The context: Why is this problem important to solve?
The objectives: What is the intended goal?
The key questions: What are the key questions that need to be answered?
The problem formulation: What are we trying to solve using data science?
⏩ Deep Learning has found applications in many predictive tasks relating to more unstructured forms of data over the last few years, such as images, text, audio and video. Many of these tasks seem to be in the vein of a larger direction of predictive modeling that aims to match human-level performance on such tasks, because humans have evolved to specialize in performing intelligent actions on such unstructured data. As a specific branch of AI (also called Affective Computing or Emotion AI) Artificial Emotional Intelligence stands for the study and development of technologies and computers that can read human emotions by means of analyzing body gestures, facial expressions, voice tone, etc. and react appropriately to them.
In the field of human-machine interaction, facial expression recognition is critical. From recent research, it has been found that as much as 55% of communication of sentiment takes place through facial expressions and other visual cues. Therefore, training a model to identify facial emotions accurately is an important step towards the development of emotionally intelligent behavior in machines with AI capabilities. Automatic facial expression recognition systems could have many applications, including but not limited to any use case that requires human behavior understanding, detection of mental disorders, and creating a higher quality of virtual assistant for customer-facing businesses.
⏩ The goal of this project is to use Deep Learning and Artificial Intelligence techniques to create a computer vision model that can accurately detect facial emotions. The model should be able to perform multi-class classification on images of facial expressions, to classify the expressions according to the associated emotion.
⏩ The data set consists of 3 folders, i.e., 'test', 'train', and 'validation'. Each of these folders has four subfolders:
‘happy’: Images of people who have happy facial expressions.
‘sad’: Images of people with sad or upset facial expressions.
‘surprise’: Images of people who have shocked or surprised facial expressions.
‘neutral’: Images of people showing no prominent emotion in their facial expression at all.
NOTE: Please use Google Colab from your browser for this notebook. Google.colab is NOT a library that can be downloaded locally on your device.
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
!ls '/content/drive/MyDrive/Colab Notebooks/Capstone/data/'
Facial_emotion_images.zip
# Importing data support libraries, including graph support
import math
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import operator
import os
import pandas as pd
import pickle
import random
import seaborn as sns
import zipfile
# Importing Machine Learning utilities
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import f1_score
from sklearn.metrics import accuracy_score
# Importing Deep Learning Libraries
import tensorflow as tf
import tensorflow.keras.applications as ap
from tensorflow.keras import backend
from tensorflow.keras.layers import Activation
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Embedding
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import GlobalAveragePooling2D
from tensorflow.keras.layers import Input
from tensorflow.keras.layers import LeakyReLU
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Reshape
from tensorflow.keras.losses import categorical_crossentropy
from tensorflow.keras.models import Model
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.utils import to_categorical
from keras.applications.vgg16 import VGG16
from keras.callbacks import EarlyStopping
from keras.callbacks import ModelCheckpoint
from keras.callbacks import ReduceLROnPlateau
# For environment checklist
from psutil import virtual_memory
# Reviewing TensorFlow library version
print('TF version: ', tf.__version__)
try:
tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection
print('Running on TPU ', tpu.cluster_spec().as_dict()['worker'])
except:
print('ERROR: Not connected to a TPU runtime')
TF version: 2.12.0 Running on TPU ['10.30.87.242:8470']
# Reviewing the memory available
ram_gb = virtual_memory().total / 1e9
print('Runtime has {:.1f} gigabytes of available RAM'.format(ram_gb))
if ram_gb < 20:
print('Not using a high-RAM runtime')
else:
print('Using a high-RAM runtime!')
Runtime has 37.8 gigabytes of available RAM Using a high-RAM runtime!
Note:
file = '/content/drive/MyDrive/Colab Notebooks/Capstone/data/Facial_emotion_images.zip'
# The data is provided as a zip file so we need to extract it from the zip file
with zipfile.ZipFile(file, 'r') as zip_ref:
zip_ref.extractall()
# Global variables
IMG_DIR = '/content/Facial_emotion_images'
MODEL_DIR = f'/content/drive/MyDrive/Colab Notebooks/Capstone/model'
TRAIN_PATH = IMG_DIR + '/train'
TEST_PATH = IMG_DIR + '/test'
VAL_PATH = IMG_DIR + '/validation'
SEED = 0
FACE_EXPRESSIONS = ['happy', 'neutral', 'sad', 'surprise']
Let's look at our classes.
Write down your observation for each class. What do you think can be a unique feature of each emotion, that separates it from the remaining classes?
def reading_files(path, name_of_set, show_sample_images=True, pick_random=True):
"""Read files in the directory path.
It can shows random sample of images if it is desired by setting
show_sample_images and pick_random parameters.
"""
files_to_read = os.listdir(path)
print(f'Total files in {name_of_set}: {len(files_to_read)}')
if show_sample_images:
fig = plt.figure(figsize=(10, 2))
if pick_random:
images = [os.path.join(path,
files_to_read[random.randint(0, len(files_to_read)-1)])
for i in range(8)]
else:
images = [os.path.join(path, file) for file in sorted(files_to_read)[-8:]]
for i, image in enumerate(images, 1):
ax = plt.subplot(1, 8, i)
ax.axis('Off')
plt.imshow(load_img(image))
plt.title(image.split('/')[-1], fontsize=6)
fig.suptitle(name_of_set, fontsize=8)
plt.tight_layout()
plt.show()
return files_to_read
def read_and_display_images_in_set(path, set_name, show_sample_images=True, pick_random=True):
"""Read over different subfolders (categories) in the directory path.
"""
files_in_set = {}
expressions = FACE_EXPRESSIONS
feature = []
for expression in expressions:
files_path = f'{path}/{expression}'
name_of_set = f'{set_name} - {expression.upper()} faces'
files_in_set[expression] = reading_files(files_path, name_of_set, show_sample_images, pick_random)
feature += [expression] * len(files_in_set[expression])
print(f'Total images: {len(feature)}')
return files_in_set, pd.Series(feature)
def get_feature_countplot(feature, set_name):
"""Countplot of a categorical variable with quantity label on each catego.
"""
sns.set(font_scale=.75)
total = len(feature) # Length of the column
plt.figure(figsize=(10, 3))
# Convert the column to a categorical data type
feature = feature.astype('category')
origin = feature.copy()
labels = feature.value_counts().index
ax = sns.countplot(x=feature, palette='Paired', order=labels)
ax.set_xlabel('')
# custom label calculates percent and add an empty string so 0 value bars don't have a number
for container in ax.containers:
labels = [f'{h:.0f}\n( {h/origin.count()*100:0.1f}% )'
if (h := v.get_height()) > 0 else '' for v in container]
ax.bar_label(container, labels=labels, label_type='edge',
fontsize='small', weight='bold') # color='white', label_type='center'
ylim = plt.ylim()
plt.ylim(ylim[0], ylim[1]*1.1)
plt.title(f'Data Distribution in {set_name}')
plt.show()
plt.style.use('default')
Training Set
title_of_set = 'Train Set'
training_set, feature_train = read_and_display_images_in_set(TRAIN_PATH, title_of_set)
Total files in Train Set - HAPPY faces: 3976
Total files in Train Set - NEUTRAL faces: 3978
Total files in Train Set - SAD faces: 3982
Total files in Train Set - SURPRISE faces: 3173
Total images: 15109
get_feature_countplot(feature_train, title_of_set)
Testing Set
title_of_set = 'Test Set'
testing_set, feature_test = read_and_display_images_in_set(TEST_PATH, title_of_set, pick_random=False)
Total files in Test Set - HAPPY faces: 32
Total files in Test Set - NEUTRAL faces: 32
Total files in Test Set - SAD faces: 32
Total files in Test Set - SURPRISE faces: 32
Total images: 128
get_feature_countplot(feature_test, title_of_set)
Validation Set
title_of_set = 'Validation Set'
validation_set, feature_val = read_and_display_images_in_set(VAL_PATH, title_of_set)
Total files in Validation Set - HAPPY faces: 1825
Total files in Validation Set - NEUTRAL faces: 1216
Total files in Validation Set - SAD faces: 1139
Total files in Validation Set - SURPRISE faces: 797
Total images: 4977
get_feature_countplot(feature_val, title_of_set)
⏩ Observations and Insights:
fig = plt.figure(figsize=(6, 2))
ax = plt.subplot(1, 2, 1)
ax.axis('Off')
plt.imshow(load_img(f'{TRAIN_PATH}/neutral/32222.jpg'))
plt.title('Trainning Set \nNeutral \n32222.jpg', fontsize=9)
ax = plt.subplot(1, 2, 2)
ax.axis('Off')
plt.imshow(load_img(f'{VAL_PATH}/happy/9984.jpg'))
plt.title('Validation Set \nHappy \n9984.jpg', fontsize=9)
plt.tight_layout()
plt.show()
# Deleting the wrong images
!rm '/content/Facial_emotion_images/test/happy/15838.jpg'
# Recalculating the testing_set and feature_test
title_of_set = 'Test Set'
testing_set, feature_test = read_and_display_images_in_set(TEST_PATH, title_of_set, show_sample_images=False)
Total files in Test Set - HAPPY faces: 31 Total files in Test Set - NEUTRAL faces: 32 Total files in Test Set - SAD faces: 32 Total files in Test Set - SURPRISE faces: 32 Total images: 127
# Observing the dimenssions of images in each set
def display_dimmension_images_in_set(path, data_set, set_name):
"""Display a scatterplot of images dimmension in the set
"""
dim = []
for expression, images in data_set.items():
files_path = f'{path}/{expression}'
dim += [(img_to_array(load_img(f'{files_path}/{image}')).shape) for image in images]
if len(set(map(operator.itemgetter(-1), dim))) == 1:
print(f'All elements in "{expression}" category has {dim[0][-1]} dimmensions')
x = list(map(operator.itemgetter(0), dim))
y = list(map(operator.itemgetter(1), dim))
fig = plt.figure(figsize=(6, 2))
plt.scatter(x,y)
plt.title(set_name)
plt.show()
display_dimmension_images_in_set(TRAIN_PATH, training_set, 'Training Set')
All elements in "happy" category has 3 dimmensions All elements in "neutral" category has 3 dimmensions All elements in "sad" category has 3 dimmensions All elements in "surprise" category has 3 dimmensions
display_dimmension_images_in_set(TEST_PATH, testing_set, 'Testing Set')
All elements in "happy" category has 3 dimmensions All elements in "neutral" category has 3 dimmensions All elements in "sad" category has 3 dimmensions All elements in "surprise" category has 3 dimmensions
display_dimmension_images_in_set(VAL_PATH, validation_set, 'Validation Set')
All elements in "happy" category has 3 dimmensions All elements in "neutral" category has 3 dimmensions All elements in "sad" category has 3 dimmensions All elements in "surprise" category has 3 dimmensions
Think About It:
⏩ Observations and Insights:
In this section, we are creating data loaders that we will use as inputs to our Neural Network.
You have two options for the color_mode. You can set it to color_mode = 'rgb' or color_mode = 'grayscale'. You will need to try out both and see for yourself which one gives better performance.
def create_data_loaders(path, color_mode, batch_size=64):
"""Create the data loaders for training, testing and validation sets.
It will be necessary to definte the color mode: rgb or grayscale.
"""
# All images to be rescaled by 1/255.
datagen_train = ImageDataGenerator(horizontal_flip = True,
rescale=1./255,
shear_range=0.3)
datagen_test = ImageDataGenerator(rescale = 1.0/255.)
datagen_val = ImageDataGenerator(rescale = 1.0/255.)
# Generating the batches od the images
default_params = {
'target_size': (48, 48),
'color_mode' : color_mode,
'class_mode' : 'categorical',
'classes' : FACE_EXPRESSIONS,
'shuffle' : True
}
print(f'\nData with color_mode={color_mode}:')
train_generator = datagen_train.flow_from_directory(f'{path}/train', batch_size=batch_size, **default_params)
test_generator = datagen_train.flow_from_directory(f'{path}/test', batch_size=32, **default_params)
val_generator = datagen_train.flow_from_directory(f'{path}/validation', batch_size=batch_size, **default_params)
return train_generator, test_generator, val_generator
train_generator_rgb, _, _ = create_data_loaders(IMG_DIR, 'rgb')
# Taking a look at some examples of our augmented training data with color_mode='rgb'.
images, labels = next(train_generator_rgb)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image)
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=rgb: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
train_generator_gray, _, _ = create_data_loaders(IMG_DIR, 'grayscale')
# Taking a look at some examples of our augmented training data with color_mode='grayscale'.
images, labels = next(train_generator_gray)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image)
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
# Using proper cmap
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image, cmap='gray')
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=grayscale: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
def save_object(obj, filename):
"""To save objects that can be loaded later"""
with open(filename, 'wb') as f:
pickle.dump(obj, f, pickle.HIGHEST_PROTOCOL)
def metrics_score(actual, predicted, model_name, history):
"""Calculating metrics to later compare performance of the model.
It also prints the classification report and the confusion matrix
"""
print(classification_report(actual, predicted, zero_division=0, target_names=FACE_EXPRESSIONS))
cm = confusion_matrix(actual, predicted)
plt.figure(figsize = (8, 5))
sns.heatmap(cm, annot=True, fmt='.0f', xticklabels=FACE_EXPRESSIONS, yticklabels=FACE_EXPRESSIONS)
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()
# Creating a dataframe of metrics
index = np.argmax(history.history['val_accuracy'])
df = pd.DataFrame(
data={
"Accuracy Train": history.history['accuracy'][index],
"Accuracy Val": history.history['val_accuracy'][index],
"Accuracy Test": accuracy_score(actual, predicted)
},
index=[model_name],
)
return df
def plot_model_result(result, model_name):
""" Plot the accuracy and loss values of the model.
"""
fig = plt.figure(figsize = (10, 3))
# Plotting the Accuracy
plt.subplot(1, 2, 1)
plt.plot(result.history['accuracy'])
plt.plot(result.history['val_accuracy'])
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['train', 'validation'], loc='upper left')
# Plotting the Loss
plt.subplot(1, 2, 2)
plt.plot(result.history['loss'])
plt.plot(result.history['val_loss'])
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['train', 'validation'], loc='upper left')
fig.suptitle(model_name)
plt.show()
def init_environment():
# Clearing previous TF sessions
backend.clear_session()
# Fixing the seed for random number generators
os.environ['PYTHONHASHSEED'] = str(2)
np.random.seed(SEED)
random.seed(SEED)
tf.random.set_seed(SEED)
print('Initializing TF Session and Seed')
⏩ Steps to follow in the model building:
def model_building_and_evaluating_process(model_builder, model_base_name, batch_size=64,
include_grayscale=True, include_rgb=True, epochs=20, tune=0):
"""Execute the following steps: (1) Initialize the environment, (2) Build and compile the model based on the
model builder method, (3) Train the model on the trainning set, and (4) Evaluate the model on the testing set
"""
model_sets = []
if include_rgb:
model_sets.append(('RGB', (48, 48, 3)))
if include_grayscale:
model_sets.append(('GRAY', (48, 48, 1)))
model_metrics = pd.DataFrame({})
model_history = {}
fitted_model = {}
for img_set, input_shape in model_sets:
model_name = f'{model_base_name} - {img_set}'
print('---------------------------------------------')
print(model_name)
print('---------------------------------------------')
# Step 1: Initialize the environment
init_environment()
train_set, test_set, val_set = create_data_loaders(IMG_DIR, color_mode='rgb' if img_set=='RGB' else 'grayscale',
batch_size=batch_size)
X_test, y_test = test_set.next()
print('\n\n')
# Step 2: Build and compile the model
if tune:
model = model_builder(input_shape=input_shape, tune=tune)
else:
model = model_builder(input_shape=input_shape)
print('\n\n')
# Step 3: Train the model on the training set
checkpoint = ModelCheckpoint(f'{MODEL_DIR}/{model_name}.h5', monitor='val_accuracy', verbose=0, save_best_only=True, mode='max')
early_stopping = EarlyStopping(monitor='val_loss', min_delta=0, patience=5, verbose=0, restore_best_weights=True)
reduce_learningrate = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3, verbose=0, min_delta=0.0001)
history = model.fit(train_set,
validation_data=val_set,
epochs=epochs,
callbacks=[early_stopping, checkpoint, reduce_learningrate])
print('\n\n')
# Step 4: Evaluate the model on the testing set
model.evaluate(X_test, y_test, verbose=1)
y_pred = np.argmax(model.predict(X_test), axis=1)
y_test = np.argmax(y_test, axis=1)
print('Actual :', y_test)
print('Prediction:', y_pred)
print('\n\n')
results = metrics_score(y_test, y_pred, model_name, history)
model_metrics = pd.concat([model_metrics, results])
print('\n\n')
fitted_model[model_name] = model
model_history[model_name] = history
plot_model_result(history, model_name)
print('\n\n')
return fitted_model, model_metrics, model_history
def loss_and_accuracy_comparisson(model_history, data_type='Training and Validation Set'):
"""Visualizing the loss and the accuracy on the train and the
validation data for all executed models.
The "data_type" to plot could be "Training and Validation Set", "Validation Set", "Training Set".
"""
fig, (ax1, ax2) = plt.subplots(1, 2, figsize = (10, 3))
for model_name, history in model_history.items():
if data_type in ["Training and Validation Set", "Training Set"]:
ax1.plot(history.history['accuracy'], ls='-', lw=1, label=f'{model_name}')
if data_type in ["Training and Validation Set", "Validation Set"]:
ax1.plot(history.history['val_accuracy'], ls='--', lw=1, label=f'{model_name} val')
ax1.set_ylabel('Accuracy', fontsize=8)
ax1.set_xlabel('Epochs', fontsize=8)
ax1.legend(fontsize=6)
ax1.set_title('Accuracy Comparisson', fontsize=8)
ax1.xaxis.set_tick_params(labelsize=7)
ax1.yaxis.set_tick_params(labelsize=7)
if data_type in ["Training and Validation Set", "Training Set"]:
ax2.plot(history.history['loss'], ls='-', lw=1, label=f'{model_name}')
if data_type in ["Training and Validation Set", "Validation Set"]:
ax2.plot(history.history['val_loss'], ls='--', lw=1, label=f'{model_name} val')
ax2.set_ylabel('Loss', fontsize=8)
ax2.set_xlabel('Epochs', fontsize=8)
ax2.legend(fontsize=6)
ax2.set_title('Loss Comparisson', fontsize=8)
ax2.xaxis.set_tick_params(labelsize=7)
ax2.yaxis.set_tick_params(labelsize=7)
fig.suptitle(f'Model Comparisson - {data_type}', fontsize=10)
plt.tight_layout()
plt.show()
Think About It:
⏩ Insights:
# Model 1: Base Neural Network
def model_builder_base_cnn(input_shape):
"""Creating the Base Neural Network.
"""
model = Sequential([
Conv2D(filters=64, kernel_size=2, activation='relu', padding='same', input_shape=input_shape),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=32, kernel_size=2, activation='relu', padding='same'),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=32, kernel_size=2, activation='relu', padding='same'),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Flatten(),
Dense(512, activation='relu'),
Dropout(0.4),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_base_cnn, metrics_base_cnn, history_base_cnn = model_building_and_evaluating_process(
model_builder_base_cnn, f'Model 1: Base Neural Network ({batch_size})', batch_size=batch_size, epochs=20
)
---------------------------------------------
Model 1: Base Neural Network (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,572
Trainable params: 605,572
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 28s 56ms/step - loss: 1.3195 - accuracy: 0.3512 - val_loss: 1.1722 - val_accuracy: 0.5055 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 26s 55ms/step - loss: 1.1272 - accuracy: 0.5053 - val_loss: 1.0094 - val_accuracy: 0.5763 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 26s 55ms/step - loss: 1.0300 - accuracy: 0.5603 - val_loss: 0.9365 - val_accuracy: 0.6156 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 26s 55ms/step - loss: 0.9791 - accuracy: 0.5833 - val_loss: 0.8959 - val_accuracy: 0.6381 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 26s 54ms/step - loss: 0.9427 - accuracy: 0.6008 - val_loss: 0.8576 - val_accuracy: 0.6426 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 26s 55ms/step - loss: 0.9150 - accuracy: 0.6117 - val_loss: 0.8524 - val_accuracy: 0.6460 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 26s 54ms/step - loss: 0.8823 - accuracy: 0.6273 - val_loss: 0.8281 - val_accuracy: 0.6558 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 26s 54ms/step - loss: 0.8707 - accuracy: 0.6345 - val_loss: 0.8111 - val_accuracy: 0.6628 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 25s 53ms/step - loss: 0.8465 - accuracy: 0.6435 - val_loss: 0.9083 - val_accuracy: 0.6211 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 26s 54ms/step - loss: 0.8320 - accuracy: 0.6503 - val_loss: 0.7770 - val_accuracy: 0.6815 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 25s 54ms/step - loss: 0.8169 - accuracy: 0.6613 - val_loss: 0.7744 - val_accuracy: 0.6795 - lr: 0.0010
Epoch 12/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7962 - accuracy: 0.6653 - val_loss: 0.7571 - val_accuracy: 0.6908 - lr: 0.0010
Epoch 13/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7925 - accuracy: 0.6695 - val_loss: 0.7417 - val_accuracy: 0.6934 - lr: 0.0010
Epoch 14/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7785 - accuracy: 0.6754 - val_loss: 0.7430 - val_accuracy: 0.6952 - lr: 0.0010
Epoch 15/20
473/473 [==============================] - 25s 53ms/step - loss: 0.7664 - accuracy: 0.6851 - val_loss: 0.7474 - val_accuracy: 0.6946 - lr: 0.0010
Epoch 16/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7591 - accuracy: 0.6842 - val_loss: 0.7310 - val_accuracy: 0.7103 - lr: 0.0010
Epoch 17/20
473/473 [==============================] - 25s 53ms/step - loss: 0.7407 - accuracy: 0.6998 - val_loss: 0.7458 - val_accuracy: 0.6922 - lr: 0.0010
Epoch 18/20
473/473 [==============================] - 25s 54ms/step - loss: 0.7391 - accuracy: 0.6977 - val_loss: 0.7134 - val_accuracy: 0.7153 - lr: 0.0010
Epoch 19/20
473/473 [==============================] - 25s 53ms/step - loss: 0.7329 - accuracy: 0.6989 - val_loss: 0.7137 - val_accuracy: 0.7137 - lr: 0.0010
Epoch 20/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7171 - accuracy: 0.7059 - val_loss: 0.7141 - val_accuracy: 0.7191 - lr: 0.0010
1/1 [==============================] - 0s 33ms/step - loss: 0.6004 - accuracy: 0.7812
1/1 [==============================] - 0s 111ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 1 3 2 2 3 0 3 2 0 2 2 3 2 2 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.80 0.89 10
neutral 0.43 0.60 0.50 5
sad 0.73 0.80 0.76 10
surprise 1.00 0.86 0.92 7
accuracy 0.78 32
macro avg 0.79 0.76 0.77 32
weighted avg 0.83 0.78 0.80 32
---------------------------------------------
Model 1: Base Neural Network (32) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,060
Trainable params: 605,060
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 18s 36ms/step - loss: 1.2927 - accuracy: 0.3759 - val_loss: 1.1302 - val_accuracy: 0.5214 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 17s 35ms/step - loss: 1.1190 - accuracy: 0.5128 - val_loss: 1.0353 - val_accuracy: 0.5712 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 17s 35ms/step - loss: 1.0362 - accuracy: 0.5545 - val_loss: 0.9599 - val_accuracy: 0.5931 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 17s 35ms/step - loss: 0.9930 - accuracy: 0.5734 - val_loss: 0.9279 - val_accuracy: 0.6078 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 17s 36ms/step - loss: 0.9548 - accuracy: 0.5902 - val_loss: 0.8821 - val_accuracy: 0.6331 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 17s 35ms/step - loss: 0.9330 - accuracy: 0.6047 - val_loss: 0.8614 - val_accuracy: 0.6458 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 17s 35ms/step - loss: 0.9047 - accuracy: 0.6176 - val_loss: 0.8477 - val_accuracy: 0.6552 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 17s 36ms/step - loss: 0.8879 - accuracy: 0.6253 - val_loss: 0.8345 - val_accuracy: 0.6566 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 16s 35ms/step - loss: 0.8648 - accuracy: 0.6349 - val_loss: 0.8628 - val_accuracy: 0.6510 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 17s 35ms/step - loss: 0.8520 - accuracy: 0.6444 - val_loss: 0.7941 - val_accuracy: 0.6789 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 16s 35ms/step - loss: 0.8362 - accuracy: 0.6494 - val_loss: 0.7748 - val_accuracy: 0.6781 - lr: 0.0010
Epoch 12/20
473/473 [==============================] - 17s 35ms/step - loss: 0.8236 - accuracy: 0.6538 - val_loss: 0.7743 - val_accuracy: 0.6805 - lr: 0.0010
Epoch 13/20
473/473 [==============================] - 17s 35ms/step - loss: 0.8113 - accuracy: 0.6628 - val_loss: 0.7691 - val_accuracy: 0.6827 - lr: 0.0010
Epoch 14/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7949 - accuracy: 0.6744 - val_loss: 0.7791 - val_accuracy: 0.6783 - lr: 0.0010
Epoch 15/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7895 - accuracy: 0.6734 - val_loss: 0.7701 - val_accuracy: 0.6821 - lr: 0.0010
Epoch 16/20
473/473 [==============================] - 16s 35ms/step - loss: 0.7774 - accuracy: 0.6793 - val_loss: 0.7605 - val_accuracy: 0.6960 - lr: 0.0010
Epoch 17/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7618 - accuracy: 0.6881 - val_loss: 0.7565 - val_accuracy: 0.6890 - lr: 0.0010
Epoch 18/20
473/473 [==============================] - 16s 35ms/step - loss: 0.7623 - accuracy: 0.6848 - val_loss: 0.7480 - val_accuracy: 0.7022 - lr: 0.0010
Epoch 19/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7532 - accuracy: 0.6906 - val_loss: 0.7451 - val_accuracy: 0.7010 - lr: 0.0010
Epoch 20/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7371 - accuracy: 0.6977 - val_loss: 0.7562 - val_accuracy: 0.6924 - lr: 0.0010
1/1 [==============================] - 0s 27ms/step - loss: 0.7368 - accuracy: 0.7812
1/1 [==============================] - 0s 81ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 1 3 2 1 0 2 1 3 2 2 3 0 3 2 0 2 2 2 1 2 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.80 0.89 10
neutral 0.44 0.80 0.57 5
sad 0.80 0.80 0.80 10
surprise 1.00 0.71 0.83 7
accuracy 0.78 32
macro avg 0.81 0.78 0.77 32
weighted avg 0.85 0.78 0.80 32
batch_size = 64
model, results, history = model_building_and_evaluating_process(model_builder_base_cnn, f'Model 1: Base Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_base_cnn.update(model)
metrics_base_cnn = pd.concat([metrics_base_cnn, results])
history_base_cnn.update(history)
---------------------------------------------
Model 1: Base Neural Network (64) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,572
Trainable params: 605,572
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 26s 105ms/step - loss: 1.3350 - accuracy: 0.3336 - val_loss: 1.2285 - val_accuracy: 0.4406 - lr: 0.0010
Epoch 2/20
237/237 [==============================] - 25s 104ms/step - loss: 1.1514 - accuracy: 0.4954 - val_loss: 1.0440 - val_accuracy: 0.5630 - lr: 0.0010
Epoch 3/20
237/237 [==============================] - 25s 104ms/step - loss: 1.0589 - accuracy: 0.5458 - val_loss: 0.9748 - val_accuracy: 0.5929 - lr: 0.0010
Epoch 4/20
237/237 [==============================] - 25s 104ms/step - loss: 1.0048 - accuracy: 0.5693 - val_loss: 0.9210 - val_accuracy: 0.6174 - lr: 0.0010
Epoch 5/20
237/237 [==============================] - 25s 104ms/step - loss: 0.9633 - accuracy: 0.5898 - val_loss: 0.8824 - val_accuracy: 0.6305 - lr: 0.0010
Epoch 6/20
237/237 [==============================] - 25s 105ms/step - loss: 0.9483 - accuracy: 0.5953 - val_loss: 0.8572 - val_accuracy: 0.6438 - lr: 0.0010
Epoch 7/20
237/237 [==============================] - 25s 104ms/step - loss: 0.9134 - accuracy: 0.6138 - val_loss: 0.8352 - val_accuracy: 0.6582 - lr: 0.0010
Epoch 8/20
237/237 [==============================] - 24s 102ms/step - loss: 0.8938 - accuracy: 0.6241 - val_loss: 0.8455 - val_accuracy: 0.6524 - lr: 0.0010
Epoch 9/20
237/237 [==============================] - 25s 104ms/step - loss: 0.8755 - accuracy: 0.6327 - val_loss: 0.8110 - val_accuracy: 0.6707 - lr: 0.0010
Epoch 10/20
237/237 [==============================] - 24s 103ms/step - loss: 0.8609 - accuracy: 0.6434 - val_loss: 0.8108 - val_accuracy: 0.6631 - lr: 0.0010
Epoch 11/20
237/237 [==============================] - 24s 103ms/step - loss: 0.8399 - accuracy: 0.6482 - val_loss: 0.8028 - val_accuracy: 0.6705 - lr: 0.0010
Epoch 12/20
237/237 [==============================] - 25s 104ms/step - loss: 0.8305 - accuracy: 0.6559 - val_loss: 0.7610 - val_accuracy: 0.6892 - lr: 0.0010
Epoch 13/20
237/237 [==============================] - 24s 102ms/step - loss: 0.8115 - accuracy: 0.6662 - val_loss: 0.7873 - val_accuracy: 0.6729 - lr: 0.0010
Epoch 14/20
237/237 [==============================] - 25s 104ms/step - loss: 0.8060 - accuracy: 0.6654 - val_loss: 0.7576 - val_accuracy: 0.6902 - lr: 0.0010
Epoch 15/20
237/237 [==============================] - 25s 104ms/step - loss: 0.7871 - accuracy: 0.6735 - val_loss: 0.7419 - val_accuracy: 0.6998 - lr: 0.0010
Epoch 16/20
237/237 [==============================] - 24s 103ms/step - loss: 0.7744 - accuracy: 0.6811 - val_loss: 0.7882 - val_accuracy: 0.6783 - lr: 0.0010
Epoch 17/20
237/237 [==============================] - 24s 102ms/step - loss: 0.7680 - accuracy: 0.6866 - val_loss: 0.7927 - val_accuracy: 0.6757 - lr: 0.0010
Epoch 18/20
237/237 [==============================] - 24s 102ms/step - loss: 0.7544 - accuracy: 0.6879 - val_loss: 0.7602 - val_accuracy: 0.6854 - lr: 0.0010
Epoch 19/20
237/237 [==============================] - 25s 104ms/step - loss: 0.7134 - accuracy: 0.7036 - val_loss: 0.7214 - val_accuracy: 0.7054 - lr: 2.0000e-04
Epoch 20/20
237/237 [==============================] - 25s 103ms/step - loss: 0.7070 - accuracy: 0.7113 - val_loss: 0.7120 - val_accuracy: 0.7133 - lr: 2.0000e-04
1/1 [==============================] - 0s 30ms/step - loss: 0.6727 - accuracy: 0.7188
1/1 [==============================] - 0s 391ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 2 3 2 0 3 0 3 2 0 2 2 3 1 2 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.38 0.60 0.46 5
sad 0.67 0.60 0.63 10
surprise 1.00 0.86 0.92 7
accuracy 0.72 32
macro avg 0.73 0.71 0.71 32
weighted avg 0.76 0.72 0.73 32
---------------------------------------------
Model 1: Base Neural Network (64) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,060
Trainable params: 605,060
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 17s 68ms/step - loss: 1.3277 - accuracy: 0.3409 - val_loss: 1.1836 - val_accuracy: 0.4762 - lr: 0.0010
Epoch 2/20
237/237 [==============================] - 16s 67ms/step - loss: 1.1535 - accuracy: 0.4954 - val_loss: 1.0556 - val_accuracy: 0.5644 - lr: 0.0010
Epoch 3/20
237/237 [==============================] - 16s 67ms/step - loss: 1.0683 - accuracy: 0.5415 - val_loss: 0.9965 - val_accuracy: 0.5857 - lr: 0.0010
Epoch 4/20
237/237 [==============================] - 16s 67ms/step - loss: 1.0169 - accuracy: 0.5640 - val_loss: 0.9461 - val_accuracy: 0.6102 - lr: 0.0010
Epoch 5/20
237/237 [==============================] - 16s 67ms/step - loss: 0.9744 - accuracy: 0.5848 - val_loss: 0.9253 - val_accuracy: 0.6225 - lr: 0.0010
Epoch 6/20
237/237 [==============================] - 16s 67ms/step - loss: 0.9466 - accuracy: 0.5993 - val_loss: 0.8794 - val_accuracy: 0.6303 - lr: 0.0010
Epoch 7/20
237/237 [==============================] - 16s 67ms/step - loss: 0.9149 - accuracy: 0.6150 - val_loss: 0.8528 - val_accuracy: 0.6522 - lr: 0.0010
Epoch 8/20
237/237 [==============================] - 16s 66ms/step - loss: 0.8908 - accuracy: 0.6288 - val_loss: 0.8313 - val_accuracy: 0.6532 - lr: 0.0010
Epoch 9/20
237/237 [==============================] - 16s 67ms/step - loss: 0.8729 - accuracy: 0.6351 - val_loss: 0.8082 - val_accuracy: 0.6683 - lr: 0.0010
Epoch 10/20
237/237 [==============================] - 16s 65ms/step - loss: 0.8685 - accuracy: 0.6331 - val_loss: 0.8210 - val_accuracy: 0.6546 - lr: 0.0010
Epoch 11/20
237/237 [==============================] - 16s 65ms/step - loss: 0.8466 - accuracy: 0.6473 - val_loss: 0.8030 - val_accuracy: 0.6631 - lr: 0.0010
Epoch 12/20
237/237 [==============================] - 16s 68ms/step - loss: 0.8272 - accuracy: 0.6552 - val_loss: 0.7729 - val_accuracy: 0.6868 - lr: 0.0010
Epoch 13/20
237/237 [==============================] - 16s 65ms/step - loss: 0.8248 - accuracy: 0.6583 - val_loss: 0.8256 - val_accuracy: 0.6578 - lr: 0.0010
Epoch 14/20
237/237 [==============================] - 16s 67ms/step - loss: 0.8069 - accuracy: 0.6639 - val_loss: 0.7737 - val_accuracy: 0.6878 - lr: 0.0010
Epoch 15/20
237/237 [==============================] - 16s 67ms/step - loss: 0.7949 - accuracy: 0.6746 - val_loss: 0.7562 - val_accuracy: 0.6928 - lr: 0.0010
Epoch 16/20
237/237 [==============================] - 15s 65ms/step - loss: 0.7805 - accuracy: 0.6791 - val_loss: 0.7726 - val_accuracy: 0.6809 - lr: 0.0010
Epoch 17/20
237/237 [==============================] - 15s 65ms/step - loss: 0.7762 - accuracy: 0.6820 - val_loss: 0.7845 - val_accuracy: 0.6767 - lr: 0.0010
Epoch 18/20
237/237 [==============================] - 16s 65ms/step - loss: 0.7628 - accuracy: 0.6854 - val_loss: 0.7865 - val_accuracy: 0.6735 - lr: 0.0010
Epoch 19/20
237/237 [==============================] - 16s 67ms/step - loss: 0.7250 - accuracy: 0.7030 - val_loss: 0.7362 - val_accuracy: 0.7018 - lr: 2.0000e-04
Epoch 20/20
237/237 [==============================] - 16s 67ms/step - loss: 0.7234 - accuracy: 0.7088 - val_loss: 0.7283 - val_accuracy: 0.7038 - lr: 2.0000e-04
1/1 [==============================] - 0s 28ms/step - loss: 0.7231 - accuracy: 0.6250
1/1 [==============================] - 0s 84ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 0 0 2 3 2 1 0 1 2 3 2 0 3 0 3 2 0 2 2 2 1 1 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.80 0.80 0.80 10
neutral 0.38 0.60 0.46 5
sad 0.50 0.50 0.50 10
surprise 1.00 0.57 0.73 7
accuracy 0.62 32
macro avg 0.67 0.62 0.62 32
weighted avg 0.68 0.62 0.64 32
batch_size = 128
model, results, history = model_building_and_evaluating_process(model_builder_base_cnn, f'Model 1: Base Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_base_cnn.update(model)
metrics_base_cnn = pd.concat([metrics_base_cnn, results])
history_base_cnn.update(history)
---------------------------------------------
Model 1: Base Neural Network (128) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,572
Trainable params: 605,572
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 26s 207ms/step - loss: 1.3598 - accuracy: 0.3080 - val_loss: 1.3520 - val_accuracy: 0.2773 - lr: 0.0010
Epoch 2/20
119/119 [==============================] - 24s 204ms/step - loss: 1.2317 - accuracy: 0.4331 - val_loss: 1.1257 - val_accuracy: 0.5280 - lr: 0.0010
Epoch 3/20
119/119 [==============================] - 24s 204ms/step - loss: 1.1147 - accuracy: 0.5164 - val_loss: 1.0227 - val_accuracy: 0.5728 - lr: 0.0010
Epoch 4/20
119/119 [==============================] - 24s 202ms/step - loss: 1.0479 - accuracy: 0.5464 - val_loss: 0.9704 - val_accuracy: 0.6014 - lr: 0.0010
Epoch 5/20
119/119 [==============================] - 24s 203ms/step - loss: 1.0082 - accuracy: 0.5658 - val_loss: 0.9355 - val_accuracy: 0.6180 - lr: 0.0010
Epoch 6/20
119/119 [==============================] - 24s 205ms/step - loss: 0.9609 - accuracy: 0.5912 - val_loss: 0.9181 - val_accuracy: 0.6217 - lr: 0.0010
Epoch 7/20
119/119 [==============================] - 24s 205ms/step - loss: 0.9404 - accuracy: 0.6039 - val_loss: 0.8763 - val_accuracy: 0.6371 - lr: 0.0010
Epoch 8/20
119/119 [==============================] - 24s 203ms/step - loss: 0.9191 - accuracy: 0.6122 - val_loss: 0.8453 - val_accuracy: 0.6560 - lr: 0.0010
Epoch 9/20
119/119 [==============================] - 24s 200ms/step - loss: 0.9035 - accuracy: 0.6227 - val_loss: 0.8501 - val_accuracy: 0.6540 - lr: 0.0010
Epoch 10/20
119/119 [==============================] - 24s 200ms/step - loss: 0.8799 - accuracy: 0.6290 - val_loss: 0.8724 - val_accuracy: 0.6391 - lr: 0.0010
Epoch 11/20
119/119 [==============================] - 24s 203ms/step - loss: 0.8736 - accuracy: 0.6312 - val_loss: 0.8216 - val_accuracy: 0.6739 - lr: 0.0010
Epoch 12/20
119/119 [==============================] - 24s 204ms/step - loss: 0.8619 - accuracy: 0.6380 - val_loss: 0.8065 - val_accuracy: 0.6805 - lr: 0.0010
Epoch 13/20
119/119 [==============================] - 24s 200ms/step - loss: 0.8513 - accuracy: 0.6451 - val_loss: 0.7790 - val_accuracy: 0.6805 - lr: 0.0010
Epoch 14/20
119/119 [==============================] - 24s 203ms/step - loss: 0.8334 - accuracy: 0.6513 - val_loss: 0.7787 - val_accuracy: 0.6835 - lr: 0.0010
Epoch 15/20
119/119 [==============================] - 24s 205ms/step - loss: 0.8248 - accuracy: 0.6558 - val_loss: 0.7671 - val_accuracy: 0.6928 - lr: 0.0010
Epoch 16/20
119/119 [==============================] - 24s 200ms/step - loss: 0.8166 - accuracy: 0.6628 - val_loss: 0.7877 - val_accuracy: 0.6890 - lr: 0.0010
Epoch 17/20
119/119 [==============================] - 24s 202ms/step - loss: 0.8073 - accuracy: 0.6650 - val_loss: 0.7626 - val_accuracy: 0.6952 - lr: 0.0010
Epoch 18/20
119/119 [==============================] - 24s 201ms/step - loss: 0.7965 - accuracy: 0.6692 - val_loss: 0.7757 - val_accuracy: 0.6870 - lr: 0.0010
Epoch 19/20
119/119 [==============================] - 24s 204ms/step - loss: 0.7800 - accuracy: 0.6832 - val_loss: 0.7544 - val_accuracy: 0.7006 - lr: 0.0010
Epoch 20/20
119/119 [==============================] - 24s 206ms/step - loss: 0.7760 - accuracy: 0.6832 - val_loss: 0.7409 - val_accuracy: 0.7018 - lr: 0.0010
1/1 [==============================] - 0s 27ms/step - loss: 0.6682 - accuracy: 0.7500
WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fcaa01143a0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
1/1 [==============================] - 0s 86ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 0 3 2 1 0 1 2 3 2 2 3 0 3 2 0 2 2 3 1 2 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.43 0.60 0.50 5
sad 0.70 0.70 0.70 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.75 0.74 0.74 32
weighted avg 0.78 0.75 0.76 32
---------------------------------------------
Model 1: Base Neural Network (128) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,060
Trainable params: 605,060
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 17s 134ms/step - loss: 1.3531 - accuracy: 0.3193 - val_loss: 1.3037 - val_accuracy: 0.3824 - lr: 0.0010
Epoch 2/20
119/119 [==============================] - 16s 132ms/step - loss: 1.2397 - accuracy: 0.4326 - val_loss: 1.1524 - val_accuracy: 0.5101 - lr: 0.0010
Epoch 3/20
119/119 [==============================] - 16s 132ms/step - loss: 1.1440 - accuracy: 0.5014 - val_loss: 1.0613 - val_accuracy: 0.5634 - lr: 0.0010
Epoch 4/20
119/119 [==============================] - 16s 131ms/step - loss: 1.0848 - accuracy: 0.5296 - val_loss: 1.0169 - val_accuracy: 0.5813 - lr: 0.0010
Epoch 5/20
119/119 [==============================] - 16s 131ms/step - loss: 1.0316 - accuracy: 0.5626 - val_loss: 0.9673 - val_accuracy: 0.5998 - lr: 0.0010
Epoch 6/20
119/119 [==============================] - 16s 132ms/step - loss: 0.9936 - accuracy: 0.5797 - val_loss: 0.9314 - val_accuracy: 0.6148 - lr: 0.0010
Epoch 7/20
119/119 [==============================] - 16s 132ms/step - loss: 0.9713 - accuracy: 0.5880 - val_loss: 0.9358 - val_accuracy: 0.6170 - lr: 0.0010
Epoch 8/20
119/119 [==============================] - 16s 131ms/step - loss: 0.9474 - accuracy: 0.6008 - val_loss: 0.8824 - val_accuracy: 0.6432 - lr: 0.0010
Epoch 9/20
119/119 [==============================] - 16s 131ms/step - loss: 0.9232 - accuracy: 0.6106 - val_loss: 0.8665 - val_accuracy: 0.6462 - lr: 0.0010
Epoch 10/20
119/119 [==============================] - 15s 129ms/step - loss: 0.9045 - accuracy: 0.6182 - val_loss: 0.9163 - val_accuracy: 0.6074 - lr: 0.0010
Epoch 11/20
119/119 [==============================] - 16s 131ms/step - loss: 0.9012 - accuracy: 0.6183 - val_loss: 0.8631 - val_accuracy: 0.6476 - lr: 0.0010
Epoch 12/20
119/119 [==============================] - 16s 131ms/step - loss: 0.8793 - accuracy: 0.6354 - val_loss: 0.8176 - val_accuracy: 0.6679 - lr: 0.0010
Epoch 13/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8718 - accuracy: 0.6347 - val_loss: 0.8207 - val_accuracy: 0.6635 - lr: 0.0010
Epoch 14/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8559 - accuracy: 0.6441 - val_loss: 0.8093 - val_accuracy: 0.6667 - lr: 0.0010
Epoch 15/20
119/119 [==============================] - 16s 131ms/step - loss: 0.8443 - accuracy: 0.6516 - val_loss: 0.8021 - val_accuracy: 0.6733 - lr: 0.0010
Epoch 16/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8346 - accuracy: 0.6558 - val_loss: 0.8442 - val_accuracy: 0.6586 - lr: 0.0010
Epoch 17/20
119/119 [==============================] - 15s 127ms/step - loss: 0.8406 - accuracy: 0.6487 - val_loss: 0.8018 - val_accuracy: 0.6679 - lr: 0.0010
Epoch 18/20
119/119 [==============================] - 16s 130ms/step - loss: 0.8326 - accuracy: 0.6560 - val_loss: 0.7731 - val_accuracy: 0.6866 - lr: 0.0010
Epoch 19/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8094 - accuracy: 0.6638 - val_loss: 0.7811 - val_accuracy: 0.6839 - lr: 0.0010
Epoch 20/20
119/119 [==============================] - 16s 130ms/step - loss: 0.8025 - accuracy: 0.6681 - val_loss: 0.7662 - val_accuracy: 0.6868 - lr: 0.0010
1/1 [==============================] - 0s 28ms/step - loss: 0.7159 - accuracy: 0.6875
WARNING:tensorflow:6 out of the last 6 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fca8027acb0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
1/1 [==============================] - 0s 87ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 0 3 0 3 2 2 2 2 3 1 2 0 2 1 0 3 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.33 0.40 0.36 5
sad 0.60 0.60 0.60 10
surprise 0.86 0.86 0.86 7
accuracy 0.69 32
macro avg 0.67 0.66 0.67 32
weighted avg 0.70 0.69 0.69 32
model_name = 'Model 1: Base Neural Network'
save_object(history_base_cnn, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metrics_base_cnn, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_base_cnn, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_base_cnn, data_type='Training Set')
loss_and_accuracy_comparisson(history_base_cnn, data_type='Validation Set')
metrics_base_cnn.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 1: Base Neural Network (32) - RGB | 0.705871 | 0.719108 | 0.781250 |
| Model 1: Base Neural Network (32) - GRAY | 0.684824 | 0.702230 | 0.781250 |
| Model 1: Base Neural Network (64) - RGB | 0.711298 | 0.713281 | 0.718750 |
| Model 1: Base Neural Network (64) - GRAY | 0.708783 | 0.703838 | 0.625000 |
| Model 1: Base Neural Network (128) - RGB | 0.683235 | 0.701828 | 0.750000 |
| Model 1: Base Neural Network (128) - GRAY | 0.668145 | 0.686759 | 0.687500 |
⏩ Observations and Insights:
⏩ Recommendations:
# Model 2: Enhanced Neural Network
def model_builder_enhanced_cnn(input_shape):
"""Creating an Enhanced Neural Network.
"""
model = Sequential([
Conv2D(filters=256, kernel_size=2, activation='relu', padding='same', input_shape=input_shape),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Conv2D(filters=128, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Conv2D(filters=64, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Conv2D(filters=32, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Flatten(),
Dense(512, activation='relu'),
Dense(128, activation='relu'),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.01) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_enhanced_cnn, metrics_enhanced_cnn, history_enhanced_cnn = model_building_and_evaluating_process(
model_builder_enhanced_cnn, f'Model 2: Enhanced Neural Network ({batch_size})', batch_size=batch_size, epochs=20
)
---------------------------------------------
Model 2: Enhanced Neural Network (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391,652
Trainable params: 390,692
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 69s 141ms/step - loss: 1.3893 - accuracy: 0.3587 - val_loss: 1.3292 - val_accuracy: 0.3757 - lr: 0.0100
Epoch 2/20
473/473 [==============================] - 66s 139ms/step - loss: 1.0473 - accuracy: 0.5200 - val_loss: 1.4469 - val_accuracy: 0.4061 - lr: 0.0100
Epoch 3/20
473/473 [==============================] - 66s 139ms/step - loss: 0.9427 - accuracy: 0.5763 - val_loss: 1.3244 - val_accuracy: 0.4541 - lr: 0.0100
Epoch 4/20
473/473 [==============================] - 66s 140ms/step - loss: 0.8756 - accuracy: 0.6178 - val_loss: 0.8860 - val_accuracy: 0.6295 - lr: 0.0100
Epoch 5/20
473/473 [==============================] - 66s 138ms/step - loss: 0.8234 - accuracy: 0.6549 - val_loss: 0.9004 - val_accuracy: 0.6227 - lr: 0.0100
Epoch 6/20
473/473 [==============================] - 66s 140ms/step - loss: 0.7941 - accuracy: 0.6711 - val_loss: 0.8781 - val_accuracy: 0.6309 - lr: 0.0100
Epoch 7/20
473/473 [==============================] - 66s 140ms/step - loss: 0.7566 - accuracy: 0.6875 - val_loss: 1.0392 - val_accuracy: 0.5425 - lr: 0.0100
Epoch 8/20
473/473 [==============================] - 67s 142ms/step - loss: 0.7365 - accuracy: 0.6935 - val_loss: 0.8052 - val_accuracy: 0.6683 - lr: 0.0100
Epoch 9/20
473/473 [==============================] - 66s 139ms/step - loss: 0.7253 - accuracy: 0.7033 - val_loss: 0.9822 - val_accuracy: 0.5943 - lr: 0.0100
Epoch 10/20
473/473 [==============================] - 66s 140ms/step - loss: 0.6975 - accuracy: 0.7157 - val_loss: 0.7527 - val_accuracy: 0.6962 - lr: 0.0100
Epoch 11/20
473/473 [==============================] - 65s 138ms/step - loss: 0.6905 - accuracy: 0.7151 - val_loss: 1.0202 - val_accuracy: 0.5959 - lr: 0.0100
Epoch 12/20
473/473 [==============================] - 65s 138ms/step - loss: 0.6701 - accuracy: 0.7220 - val_loss: 0.7829 - val_accuracy: 0.6759 - lr: 0.0100
Epoch 13/20
473/473 [==============================] - 66s 139ms/step - loss: 0.6630 - accuracy: 0.7287 - val_loss: 0.8674 - val_accuracy: 0.6582 - lr: 0.0100
Epoch 14/20
473/473 [==============================] - 66s 140ms/step - loss: 0.5698 - accuracy: 0.7699 - val_loss: 0.7297 - val_accuracy: 0.7215 - lr: 0.0020
Epoch 15/20
473/473 [==============================] - 66s 139ms/step - loss: 0.5381 - accuracy: 0.7824 - val_loss: 0.7504 - val_accuracy: 0.7131 - lr: 0.0020
Epoch 16/20
473/473 [==============================] - 66s 140ms/step - loss: 0.5154 - accuracy: 0.7923 - val_loss: 0.7193 - val_accuracy: 0.7271 - lr: 0.0020
Epoch 17/20
473/473 [==============================] - 67s 142ms/step - loss: 0.4999 - accuracy: 0.7931 - val_loss: 0.7519 - val_accuracy: 0.7183 - lr: 0.0020
Epoch 18/20
473/473 [==============================] - 80s 170ms/step - loss: 0.4833 - accuracy: 0.8048 - val_loss: 0.7527 - val_accuracy: 0.7189 - lr: 0.0020
Epoch 19/20
473/473 [==============================] - 79s 166ms/step - loss: 0.4647 - accuracy: 0.8126 - val_loss: 0.7593 - val_accuracy: 0.7265 - lr: 0.0020
Epoch 20/20
473/473 [==============================] - 78s 165ms/step - loss: 0.4299 - accuracy: 0.8293 - val_loss: 0.7631 - val_accuracy: 0.7255 - lr: 4.0000e-04
1/1 [==============================] - 0s 68ms/step - loss: 0.4700 - accuracy: 0.8438
1/1 [==============================] - 0s 180ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 1 2 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.90 0.95 10
neutral 0.57 0.80 0.67 5
sad 0.80 0.80 0.80 10
surprise 1.00 0.86 0.92 7
accuracy 0.84 32
macro avg 0.84 0.84 0.83 32
weighted avg 0.87 0.84 0.85 32
---------------------------------------------
Model 2: Enhanced Neural Network (32) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389,604
Trainable params: 388,644
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 78s 160ms/step - loss: 1.3877 - accuracy: 0.3525 - val_loss: 1.2700 - val_accuracy: 0.3434 - lr: 0.0100
Epoch 2/20
473/473 [==============================] - 72s 153ms/step - loss: 1.0874 - accuracy: 0.5033 - val_loss: 1.1124 - val_accuracy: 0.5079 - lr: 0.0100
Epoch 3/20
473/473 [==============================] - 74s 157ms/step - loss: 0.9372 - accuracy: 0.5949 - val_loss: 0.9057 - val_accuracy: 0.6277 - lr: 0.0100
Epoch 4/20
473/473 [==============================] - 72s 152ms/step - loss: 0.8693 - accuracy: 0.6280 - val_loss: 0.9304 - val_accuracy: 0.6130 - lr: 0.0100
Epoch 5/20
473/473 [==============================] - 69s 146ms/step - loss: 0.8381 - accuracy: 0.6477 - val_loss: 0.9043 - val_accuracy: 0.6299 - lr: 0.0100
Epoch 6/20
473/473 [==============================] - 68s 143ms/step - loss: 0.8052 - accuracy: 0.6638 - val_loss: 1.0157 - val_accuracy: 0.5843 - lr: 0.0100
Epoch 7/20
473/473 [==============================] - 67s 142ms/step - loss: 0.7746 - accuracy: 0.6803 - val_loss: 0.8872 - val_accuracy: 0.6472 - lr: 0.0100
Epoch 8/20
473/473 [==============================] - 67s 141ms/step - loss: 0.7455 - accuracy: 0.6926 - val_loss: 0.8119 - val_accuracy: 0.6723 - lr: 0.0100
Epoch 9/20
473/473 [==============================] - 67s 141ms/step - loss: 0.7258 - accuracy: 0.7029 - val_loss: 1.6141 - val_accuracy: 0.5612 - lr: 0.0100
Epoch 10/20
473/473 [==============================] - 71s 149ms/step - loss: 0.7120 - accuracy: 0.7115 - val_loss: 0.8237 - val_accuracy: 0.6558 - lr: 0.0100
Epoch 11/20
473/473 [==============================] - 69s 146ms/step - loss: 0.6929 - accuracy: 0.7186 - val_loss: 1.1831 - val_accuracy: 0.5001 - lr: 0.0100
Epoch 12/20
473/473 [==============================] - 68s 144ms/step - loss: 0.6094 - accuracy: 0.7522 - val_loss: 0.7039 - val_accuracy: 0.7085 - lr: 0.0020
Epoch 13/20
473/473 [==============================] - 68s 144ms/step - loss: 0.5724 - accuracy: 0.7721 - val_loss: 0.7011 - val_accuracy: 0.7237 - lr: 0.0020
Epoch 14/20
473/473 [==============================] - 71s 149ms/step - loss: 0.5578 - accuracy: 0.7740 - val_loss: 0.7767 - val_accuracy: 0.6958 - lr: 0.0020
Epoch 15/20
473/473 [==============================] - 71s 149ms/step - loss: 0.5400 - accuracy: 0.7831 - val_loss: 0.7318 - val_accuracy: 0.7267 - lr: 0.0020
Epoch 16/20
473/473 [==============================] - 68s 144ms/step - loss: 0.5205 - accuracy: 0.7915 - val_loss: 0.7206 - val_accuracy: 0.7314 - lr: 0.0020
Epoch 17/20
473/473 [==============================] - 69s 146ms/step - loss: 0.4907 - accuracy: 0.8020 - val_loss: 0.7196 - val_accuracy: 0.7304 - lr: 4.0000e-04
Epoch 18/20
473/473 [==============================] - 72s 152ms/step - loss: 0.4783 - accuracy: 0.8084 - val_loss: 0.7350 - val_accuracy: 0.7302 - lr: 4.0000e-04
1/1 [==============================] - 0s 65ms/step - loss: 0.6166 - accuracy: 0.8125
1/1 [==============================] - 0s 172ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 1 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 2 3 1 2 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.90 0.95 10
neutral 0.44 0.80 0.57 5
sad 0.88 0.70 0.78 10
surprise 1.00 0.86 0.92 7
accuracy 0.81 32
macro avg 0.83 0.81 0.80 32
weighted avg 0.87 0.81 0.83 32
batch_size = 64
model, results, history = model_building_and_evaluating_process(model_builder_enhanced_cnn, f'Model 2: Enhanced Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_enhanced_cnn.update(model)
metrics_enhanced_cnn = pd.concat([metrics_enhanced_cnn, results])
history_enhanced_cnn.update(history)
---------------------------------------------
Model 2: Enhanced Neural Network (64) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391,652
Trainable params: 390,692
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 71s 290ms/step - loss: 1.4343 - accuracy: 0.3738 - val_loss: 1.3023 - val_accuracy: 0.4097 - lr: 0.0100
Epoch 2/20
237/237 [==============================] - 68s 287ms/step - loss: 1.0529 - accuracy: 0.5253 - val_loss: 1.1257 - val_accuracy: 0.4959 - lr: 0.0100
Epoch 3/20
237/237 [==============================] - 66s 279ms/step - loss: 0.9480 - accuracy: 0.5859 - val_loss: 1.0692 - val_accuracy: 0.5373 - lr: 0.0100
Epoch 4/20
237/237 [==============================] - 68s 288ms/step - loss: 0.8785 - accuracy: 0.6289 - val_loss: 0.9133 - val_accuracy: 0.6080 - lr: 0.0100
Epoch 5/20
237/237 [==============================] - 66s 278ms/step - loss: 0.8345 - accuracy: 0.6466 - val_loss: 0.9580 - val_accuracy: 0.5672 - lr: 0.0100
Epoch 6/20
237/237 [==============================] - 66s 279ms/step - loss: 0.7965 - accuracy: 0.6662 - val_loss: 1.0283 - val_accuracy: 0.5393 - lr: 0.0100
Epoch 7/20
237/237 [==============================] - 66s 280ms/step - loss: 0.7560 - accuracy: 0.6842 - val_loss: 0.9096 - val_accuracy: 0.6034 - lr: 0.0100
Epoch 8/20
237/237 [==============================] - 67s 282ms/step - loss: 0.7458 - accuracy: 0.6911 - val_loss: 0.9428 - val_accuracy: 0.6223 - lr: 0.0100
Epoch 9/20
237/237 [==============================] - 67s 280ms/step - loss: 0.7233 - accuracy: 0.7022 - val_loss: 0.8124 - val_accuracy: 0.6665 - lr: 0.0100
Epoch 10/20
237/237 [==============================] - 65s 274ms/step - loss: 0.6904 - accuracy: 0.7161 - val_loss: 0.8657 - val_accuracy: 0.6345 - lr: 0.0100
Epoch 11/20
237/237 [==============================] - 64s 269ms/step - loss: 0.6786 - accuracy: 0.7200 - val_loss: 1.0323 - val_accuracy: 0.5746 - lr: 0.0100
Epoch 12/20
237/237 [==============================] - 64s 270ms/step - loss: 0.6735 - accuracy: 0.7235 - val_loss: 0.7623 - val_accuracy: 0.6914 - lr: 0.0100
Epoch 13/20
237/237 [==============================] - 64s 272ms/step - loss: 0.6468 - accuracy: 0.7343 - val_loss: 0.8515 - val_accuracy: 0.6506 - lr: 0.0100
Epoch 14/20
237/237 [==============================] - 66s 280ms/step - loss: 0.6523 - accuracy: 0.7341 - val_loss: 0.8055 - val_accuracy: 0.6735 - lr: 0.0100
Epoch 15/20
237/237 [==============================] - 67s 283ms/step - loss: 0.6228 - accuracy: 0.7449 - val_loss: 0.7623 - val_accuracy: 0.6934 - lr: 0.0100
Epoch 16/20
237/237 [==============================] - 66s 278ms/step - loss: 0.5357 - accuracy: 0.7836 - val_loss: 0.7076 - val_accuracy: 0.7179 - lr: 0.0020
Epoch 17/20
237/237 [==============================] - 67s 281ms/step - loss: 0.5034 - accuracy: 0.7944 - val_loss: 0.7176 - val_accuracy: 0.7195 - lr: 0.0020
Epoch 18/20
237/237 [==============================] - 68s 287ms/step - loss: 0.4814 - accuracy: 0.8036 - val_loss: 0.7248 - val_accuracy: 0.7197 - lr: 0.0020
Epoch 19/20
237/237 [==============================] - 65s 276ms/step - loss: 0.4638 - accuracy: 0.8125 - val_loss: 0.7596 - val_accuracy: 0.7169 - lr: 0.0020
Epoch 20/20
237/237 [==============================] - 66s 278ms/step - loss: 0.4301 - accuracy: 0.8266 - val_loss: 0.7374 - val_accuracy: 0.7368 - lr: 4.0000e-04
1/1 [==============================] - 0s 59ms/step - loss: 0.8335 - accuracy: 0.6875
1/1 [==============================] - 0s 168ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 0 0 0 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 2 2 1 1 0 2 1 0 0 1 0 2]
precision recall f1-score support
happy 0.75 0.90 0.82 10
neutral 0.33 0.40 0.36 5
sad 0.70 0.70 0.70 10
surprise 1.00 0.57 0.73 7
accuracy 0.69 32
macro avg 0.70 0.64 0.65 32
weighted avg 0.72 0.69 0.69 32
---------------------------------------------
Model 2: Enhanced Neural Network (64) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389,604
Trainable params: 388,644
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 67s 275ms/step - loss: 1.4771 - accuracy: 0.3600 - val_loss: 1.4708 - val_accuracy: 0.3221 - lr: 0.0100
Epoch 2/20
237/237 [==============================] - 66s 279ms/step - loss: 1.1109 - accuracy: 0.4953 - val_loss: 1.1091 - val_accuracy: 0.4850 - lr: 0.0100
Epoch 3/20
237/237 [==============================] - 67s 281ms/step - loss: 0.9491 - accuracy: 0.5890 - val_loss: 1.3789 - val_accuracy: 0.3982 - lr: 0.0100
Epoch 4/20
237/237 [==============================] - 67s 281ms/step - loss: 0.8670 - accuracy: 0.6362 - val_loss: 1.2985 - val_accuracy: 0.4947 - lr: 0.0100
Epoch 5/20
237/237 [==============================] - 66s 279ms/step - loss: 0.8030 - accuracy: 0.6671 - val_loss: 0.8474 - val_accuracy: 0.6480 - lr: 0.0100
Epoch 6/20
237/237 [==============================] - 65s 275ms/step - loss: 0.7823 - accuracy: 0.6764 - val_loss: 1.2658 - val_accuracy: 0.4623 - lr: 0.0100
Epoch 7/20
237/237 [==============================] - 67s 281ms/step - loss: 0.7453 - accuracy: 0.6904 - val_loss: 1.1134 - val_accuracy: 0.5017 - lr: 0.0100
Epoch 8/20
237/237 [==============================] - 67s 282ms/step - loss: 0.7310 - accuracy: 0.6989 - val_loss: 0.8642 - val_accuracy: 0.6430 - lr: 0.0100
Epoch 9/20
237/237 [==============================] - 67s 284ms/step - loss: 0.6453 - accuracy: 0.7391 - val_loss: 0.7260 - val_accuracy: 0.7060 - lr: 0.0020
Epoch 10/20
237/237 [==============================] - 65s 274ms/step - loss: 0.6114 - accuracy: 0.7509 - val_loss: 0.7420 - val_accuracy: 0.6990 - lr: 0.0020
Epoch 11/20
237/237 [==============================] - 65s 272ms/step - loss: 0.5994 - accuracy: 0.7577 - val_loss: 0.7393 - val_accuracy: 0.6996 - lr: 0.0020
Epoch 12/20
237/237 [==============================] - 66s 279ms/step - loss: 0.5818 - accuracy: 0.7635 - val_loss: 0.7339 - val_accuracy: 0.7071 - lr: 0.0020
Epoch 13/20
237/237 [==============================] - 66s 280ms/step - loss: 0.5456 - accuracy: 0.7798 - val_loss: 0.7099 - val_accuracy: 0.7219 - lr: 4.0000e-04
Epoch 14/20
237/237 [==============================] - 67s 284ms/step - loss: 0.5328 - accuracy: 0.7842 - val_loss: 0.7088 - val_accuracy: 0.7193 - lr: 4.0000e-04
Epoch 15/20
237/237 [==============================] - 69s 290ms/step - loss: 0.5242 - accuracy: 0.7880 - val_loss: 0.7054 - val_accuracy: 0.7277 - lr: 4.0000e-04
Epoch 16/20
237/237 [==============================] - 66s 278ms/step - loss: 0.5188 - accuracy: 0.7888 - val_loss: 0.7322 - val_accuracy: 0.7177 - lr: 4.0000e-04
Epoch 17/20
237/237 [==============================] - 66s 280ms/step - loss: 0.5141 - accuracy: 0.7944 - val_loss: 0.7164 - val_accuracy: 0.7167 - lr: 4.0000e-04
Epoch 18/20
237/237 [==============================] - 65s 273ms/step - loss: 0.5106 - accuracy: 0.7966 - val_loss: 0.7246 - val_accuracy: 0.7227 - lr: 4.0000e-04
Epoch 19/20
237/237 [==============================] - 65s 273ms/step - loss: 0.4973 - accuracy: 0.8042 - val_loss: 0.7274 - val_accuracy: 0.7221 - lr: 8.0000e-05
Epoch 20/20
237/237 [==============================] - 65s 276ms/step - loss: 0.4975 - accuracy: 0.7991 - val_loss: 0.7241 - val_accuracy: 0.7219 - lr: 8.0000e-05
1/1 [==============================] - 0s 59ms/step - loss: 0.5514 - accuracy: 0.7500
1/1 [==============================] - 0s 166ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 2 3 1 2 0 1 1 0 0 1 0 2]
precision recall f1-score support
happy 0.90 0.90 0.90 10
neutral 0.29 0.40 0.33 5
sad 0.78 0.70 0.74 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.74 0.71 0.72 32
weighted avg 0.79 0.75 0.77 32
batch_size = 128
model, results, history = model_building_and_evaluating_process(model_builder_enhanced_cnn, f'Model 2: Enhanced Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_enhanced_cnn.update(model)
metrics_enhanced_cnn = pd.concat([metrics_enhanced_cnn, results])
history_enhanced_cnn.update(history)
---------------------------------------------
Model 2: Enhanced Neural Network (128) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391,652
Trainable params: 390,692
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 64s 519ms/step - loss: 1.6542 - accuracy: 0.3290 - val_loss: 2.4356 - val_accuracy: 0.2767 - lr: 0.0100
Epoch 2/20
119/119 [==============================] - 61s 512ms/step - loss: 1.1971 - accuracy: 0.4397 - val_loss: 1.4953 - val_accuracy: 0.3705 - lr: 0.0100
Epoch 3/20
119/119 [==============================] - 61s 515ms/step - loss: 0.9932 - accuracy: 0.5528 - val_loss: 1.2061 - val_accuracy: 0.4422 - lr: 0.0100
Epoch 4/20
119/119 [==============================] - 64s 536ms/step - loss: 0.9217 - accuracy: 0.6010 - val_loss: 1.1810 - val_accuracy: 0.4862 - lr: 0.0100
Epoch 5/20
119/119 [==============================] - 63s 527ms/step - loss: 0.8618 - accuracy: 0.6308 - val_loss: 1.0730 - val_accuracy: 0.5421 - lr: 0.0100
Epoch 6/20
119/119 [==============================] - 62s 522ms/step - loss: 0.8079 - accuracy: 0.6564 - val_loss: 1.1524 - val_accuracy: 0.5228 - lr: 0.0100
Epoch 7/20
119/119 [==============================] - 61s 511ms/step - loss: 0.7682 - accuracy: 0.6809 - val_loss: 0.8772 - val_accuracy: 0.6488 - lr: 0.0100
Epoch 8/20
119/119 [==============================] - 62s 519ms/step - loss: 0.7431 - accuracy: 0.6910 - val_loss: 0.8538 - val_accuracy: 0.6484 - lr: 0.0100
Epoch 9/20
119/119 [==============================] - 62s 519ms/step - loss: 0.7175 - accuracy: 0.7035 - val_loss: 0.9057 - val_accuracy: 0.6367 - lr: 0.0100
Epoch 10/20
119/119 [==============================] - 61s 515ms/step - loss: 0.6923 - accuracy: 0.7154 - val_loss: 0.8409 - val_accuracy: 0.6381 - lr: 0.0100
Epoch 11/20
119/119 [==============================] - 61s 508ms/step - loss: 0.6918 - accuracy: 0.7132 - val_loss: 1.1288 - val_accuracy: 0.5558 - lr: 0.0100
Epoch 12/20
119/119 [==============================] - 61s 515ms/step - loss: 0.6682 - accuracy: 0.7259 - val_loss: 0.7400 - val_accuracy: 0.7083 - lr: 0.0100
Epoch 13/20
119/119 [==============================] - 62s 518ms/step - loss: 0.6393 - accuracy: 0.7362 - val_loss: 1.1642 - val_accuracy: 0.5188 - lr: 0.0100
Epoch 14/20
119/119 [==============================] - 62s 522ms/step - loss: 0.6471 - accuracy: 0.7364 - val_loss: 0.8443 - val_accuracy: 0.6552 - lr: 0.0100
Epoch 15/20
119/119 [==============================] - 63s 528ms/step - loss: 0.6339 - accuracy: 0.7405 - val_loss: 1.0644 - val_accuracy: 0.5449 - lr: 0.0100
Epoch 16/20
119/119 [==============================] - 65s 548ms/step - loss: 0.5522 - accuracy: 0.7772 - val_loss: 0.7175 - val_accuracy: 0.7185 - lr: 0.0020
Epoch 17/20
119/119 [==============================] - 63s 533ms/step - loss: 0.5399 - accuracy: 0.7806 - val_loss: 0.7433 - val_accuracy: 0.7203 - lr: 0.0020
Epoch 18/20
119/119 [==============================] - 63s 525ms/step - loss: 0.5165 - accuracy: 0.7950 - val_loss: 0.7341 - val_accuracy: 0.7171 - lr: 0.0020
Epoch 19/20
119/119 [==============================] - 63s 530ms/step - loss: 0.4966 - accuracy: 0.7983 - val_loss: 0.7138 - val_accuracy: 0.7292 - lr: 0.0020
Epoch 20/20
119/119 [==============================] - 63s 529ms/step - loss: 0.4891 - accuracy: 0.8049 - val_loss: 0.8095 - val_accuracy: 0.7024 - lr: 0.0020
1/1 [==============================] - 0s 58ms/step - loss: 0.7810 - accuracy: 0.7812
1/1 [==============================] - 0s 166ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 2 3 0 1 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 2 2 0 2 0 0 1 0 0 2 0 2]
precision recall f1-score support
happy 0.83 1.00 0.91 10
neutral 0.50 0.40 0.44 5
sad 0.73 0.80 0.76 10
surprise 1.00 0.71 0.83 7
accuracy 0.78 32
macro avg 0.77 0.73 0.74 32
weighted avg 0.78 0.78 0.77 32
---------------------------------------------
Model 2: Enhanced Neural Network (128) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389,604
Trainable params: 388,644
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 66s 538ms/step - loss: 1.6232 - accuracy: 0.3075 - val_loss: 1.3630 - val_accuracy: 0.3719 - lr: 0.0100
Epoch 2/20
119/119 [==============================] - 63s 531ms/step - loss: 1.3055 - accuracy: 0.3508 - val_loss: 1.3287 - val_accuracy: 0.3661 - lr: 0.0100
Epoch 3/20
119/119 [==============================] - 63s 528ms/step - loss: 1.1279 - accuracy: 0.4720 - val_loss: 1.7102 - val_accuracy: 0.2885 - lr: 0.0100
Epoch 4/20
119/119 [==============================] - 63s 530ms/step - loss: 0.9815 - accuracy: 0.5736 - val_loss: 1.1245 - val_accuracy: 0.4917 - lr: 0.0100
Epoch 5/20
119/119 [==============================] - 64s 535ms/step - loss: 0.8900 - accuracy: 0.6237 - val_loss: 1.3265 - val_accuracy: 0.4374 - lr: 0.0100
Epoch 6/20
119/119 [==============================] - 63s 527ms/step - loss: 0.8142 - accuracy: 0.6593 - val_loss: 0.9534 - val_accuracy: 0.5994 - lr: 0.0100
Epoch 7/20
119/119 [==============================] - 62s 524ms/step - loss: 0.7780 - accuracy: 0.6763 - val_loss: 0.8655 - val_accuracy: 0.6317 - lr: 0.0100
Epoch 8/20
119/119 [==============================] - 62s 524ms/step - loss: 0.7418 - accuracy: 0.6944 - val_loss: 0.9340 - val_accuracy: 0.6219 - lr: 0.0100
Epoch 9/20
119/119 [==============================] - 62s 521ms/step - loss: 0.7574 - accuracy: 0.6847 - val_loss: 0.9271 - val_accuracy: 0.6259 - lr: 0.0100
Epoch 10/20
119/119 [==============================] - 62s 522ms/step - loss: 0.7046 - accuracy: 0.7081 - val_loss: 1.6188 - val_accuracy: 0.4278 - lr: 0.0100
Epoch 11/20
119/119 [==============================] - 63s 527ms/step - loss: 0.6439 - accuracy: 0.7373 - val_loss: 0.7312 - val_accuracy: 0.7091 - lr: 0.0020
Epoch 12/20
119/119 [==============================] - 63s 530ms/step - loss: 0.6138 - accuracy: 0.7464 - val_loss: 0.7014 - val_accuracy: 0.7165 - lr: 0.0020
Epoch 13/20
119/119 [==============================] - 63s 532ms/step - loss: 0.5905 - accuracy: 0.7578 - val_loss: 0.7234 - val_accuracy: 0.7133 - lr: 0.0020
Epoch 14/20
119/119 [==============================] - 63s 529ms/step - loss: 0.5896 - accuracy: 0.7609 - val_loss: 0.7147 - val_accuracy: 0.7187 - lr: 0.0020
Epoch 15/20
119/119 [==============================] - 62s 524ms/step - loss: 0.5730 - accuracy: 0.7642 - val_loss: 0.7030 - val_accuracy: 0.7227 - lr: 0.0020
Epoch 16/20
119/119 [==============================] - 62s 524ms/step - loss: 0.5514 - accuracy: 0.7771 - val_loss: 0.7115 - val_accuracy: 0.7217 - lr: 4.0000e-04
Epoch 17/20
119/119 [==============================] - 62s 525ms/step - loss: 0.5377 - accuracy: 0.7799 - val_loss: 0.7362 - val_accuracy: 0.7153 - lr: 4.0000e-04
1/1 [==============================] - 0s 60ms/step - loss: 0.7239 - accuracy: 0.7500
1/1 [==============================] - 0s 162ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 0 3 2 1 0 1 0 3 2 0 3 0 3 2 0 2 2 3 1 1 0 0 1 0 1 0 0 2]
precision recall f1-score support
happy 0.77 1.00 0.87 10
neutral 0.43 0.60 0.50 5
sad 0.83 0.50 0.62 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.76 0.74 0.73 32
weighted avg 0.79 0.75 0.75 32
model_name = 'Model 2: Enhanced Neural Network'
save_object(history_enhanced_cnn, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metrics_enhanced_cnn, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_enhanced_cnn, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_enhanced_cnn, data_type='Training Set')
loss_and_accuracy_comparisson(history_enhanced_cnn, data_type='Validation Set')
metrics_enhanced_cnn.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 2: Enhanced Neural Network (32) - RGB | 0.792309 | 0.727145 | 0.843750 |
| Model 2: Enhanced Neural Network (32) - GRAY | 0.791515 | 0.731364 | 0.812500 |
| Model 2: Enhanced Neural Network (64) - RGB | 0.826593 | 0.736789 | 0.687500 |
| Model 2: Enhanced Neural Network (64) - GRAY | 0.788007 | 0.727748 | 0.750000 |
| Model 2: Enhanced Neural Network (128) - RGB | 0.798332 | 0.729154 | 0.781250 |
| Model 2: Enhanced Neural Network (128) - GRAY | 0.764180 | 0.722725 | 0.750000 |
⏩ Observations and Insights:
⏩ Recommendations:
Think About It:
⏩ Observations and Insights:
In this section, we will create several Transfer Learning architectures. For the pre-trained models, we will select three popular architectures namely, VGG16, ResNet v2, and Efficient Net. The difference between these architectures and the previous architectures is that these will require 3 input channels while the earlier ones worked on 'grayscale' images. Therefore, we need to create new DataLoaders.
In this section, we are creating data loaders that we will use as inputs to our Neural Network. We will have to go with color_mode = 'rgb' as this is the required format for the transfer learning architectures.
train_generator_rgb, _, _ = create_data_loaders(IMG_DIR, 'rgb')
# Taking a look at some examples of our augmented training data with color_mode='rgb'.
images, labels = next(train_generator_rgb)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image)
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=rgb: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
VGG16(weights='imagenet', include_top=False, input_shape = (48, 48, 3)).summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58889256/58889256 [==============================] - 0s 0us/step
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 48, 48, 3)] 0
block1_conv1 (Conv2D) (None, 48, 48, 64) 1792
block1_conv2 (Conv2D) (None, 48, 48, 64) 36928
block1_pool (MaxPooling2D) (None, 24, 24, 64) 0
block2_conv1 (Conv2D) (None, 24, 24, 128) 73856
block2_conv2 (Conv2D) (None, 24, 24, 128) 147584
block2_pool (MaxPooling2D) (None, 12, 12, 128) 0
block3_conv1 (Conv2D) (None, 12, 12, 256) 295168
block3_conv2 (Conv2D) (None, 12, 12, 256) 590080
block3_conv3 (Conv2D) (None, 12, 12, 256) 590080
block3_pool (MaxPooling2D) (None, 6, 6, 256) 0
block4_conv1 (Conv2D) (None, 6, 6, 512) 1180160
block4_conv2 (Conv2D) (None, 6, 6, 512) 2359808
block4_conv3 (Conv2D) (None, 6, 6, 512) 2359808
block4_pool (MaxPooling2D) (None, 3, 3, 512) 0
block5_conv1 (Conv2D) (None, 3, 3, 512) 2359808
block5_conv2 (Conv2D) (None, 3, 3, 512) 2359808
block5_conv3 (Conv2D) (None, 3, 3, 512) 2359808
block5_pool (MaxPooling2D) (None, 1, 1, 512) 0
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________
# Model 3: VGG16
def model_builder_vgg16(input_shape, tune=0):
"""Transfer approach with the VGG16 Model
"""
vgg_model = VGG16(weights='imagenet', include_top=False, input_shape = input_shape)
# Making all the layers of the VGG model non-trainable
if tune > 0:
for layer in vgg_model.layers[:-tune]:
layer.trainable = False
else:
for layer in vgg_model.layers:
layer.trainable = False
model = Sequential([
vgg_model,
Flatten(),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dropout(0.3),
Dense(64, activation='relu'),
BatchNormalization(),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_vgg16, metric_vgg16, history_vgg16 = model_building_and_evaluating_process(
model_builder_vgg16, f'Model 3: VGG16 ({batch_size})', batch_size=batch_size, epochs=20, include_grayscale=False
)
# model_vgg16['Model 3: VGG16 (32) - RGB'].get_metrics_result()['accuracy'].numpy()
---------------------------------------------
Model 3: VGG16 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 1, 1, 512) 14714688
flatten (Flatten) (None, 512) 0
dense (Dense) (None, 256) 131328
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 14,887,684
Trainable params: 172,868
Non-trainable params: 14,714,816
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 41s 83ms/step - loss: 1.2524 - accuracy: 0.4357 - val_loss: 1.0932 - val_accuracy: 0.5252 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 38s 80ms/step - loss: 1.1323 - accuracy: 0.5042 - val_loss: 1.2063 - val_accuracy: 0.4663 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 37s 79ms/step - loss: 1.0921 - accuracy: 0.5343 - val_loss: 1.1006 - val_accuracy: 0.5186 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 39s 82ms/step - loss: 1.0750 - accuracy: 0.5391 - val_loss: 1.0562 - val_accuracy: 0.5431 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 38s 80ms/step - loss: 1.0516 - accuracy: 0.5501 - val_loss: 1.0475 - val_accuracy: 0.5479 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 38s 80ms/step - loss: 1.0294 - accuracy: 0.5634 - val_loss: 1.0337 - val_accuracy: 0.5588 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 37s 79ms/step - loss: 1.0172 - accuracy: 0.5627 - val_loss: 1.1040 - val_accuracy: 0.5236 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 37s 78ms/step - loss: 1.0022 - accuracy: 0.5729 - val_loss: 1.0965 - val_accuracy: 0.5250 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 37s 78ms/step - loss: 0.9757 - accuracy: 0.5891 - val_loss: 1.0464 - val_accuracy: 0.5437 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 38s 80ms/step - loss: 0.9240 - accuracy: 0.6129 - val_loss: 1.0171 - val_accuracy: 0.5668 - lr: 2.0000e-04
Epoch 11/20
473/473 [==============================] - 37s 79ms/step - loss: 0.8960 - accuracy: 0.6282 - val_loss: 1.0418 - val_accuracy: 0.5531 - lr: 2.0000e-04
Epoch 12/20
473/473 [==============================] - 37s 78ms/step - loss: 0.8756 - accuracy: 0.6335 - val_loss: 1.0299 - val_accuracy: 0.5576 - lr: 2.0000e-04
Epoch 13/20
473/473 [==============================] - 37s 79ms/step - loss: 0.8700 - accuracy: 0.6419 - val_loss: 1.0422 - val_accuracy: 0.5622 - lr: 2.0000e-04
Epoch 14/20
473/473 [==============================] - 38s 80ms/step - loss: 0.8420 - accuracy: 0.6513 - val_loss: 1.0207 - val_accuracy: 0.5732 - lr: 4.0000e-05
Epoch 15/20
473/473 [==============================] - 37s 78ms/step - loss: 0.8319 - accuracy: 0.6550 - val_loss: 1.0324 - val_accuracy: 0.5644 - lr: 4.0000e-05
1/1 [==============================] - 0s 71ms/step - loss: 0.8450 - accuracy: 0.6250
1/1 [==============================] - 0s 225ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 2 0 1 2 3 1 1 0 2 1 3 2 2 3 0 3 2 0 2 1 3 1 2 1 2 1 0 2 1 0 2]
precision recall f1-score support
happy 0.86 0.60 0.71 10
neutral 0.22 0.40 0.29 5
sad 0.64 0.70 0.67 10
surprise 1.00 0.71 0.83 7
accuracy 0.62 32
macro avg 0.68 0.60 0.62 32
weighted avg 0.72 0.62 0.66 32
# Let's try again tuning some of the last layers in the transfer model
batch_size = 32
model, results, history = model_building_and_evaluating_process(model_builder_vgg16, f'Model 3: Tuned VGG16 ({batch_size})',
batch_size=batch_size, epochs=20, tune=4, include_grayscale=False)
model_vgg16.update(model)
metric_vgg16 = pd.concat([metric_vgg16, results])
history_vgg16.update(history)
---------------------------------------------
Model 3: Tuned VGG16 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 1, 1, 512) 14714688
flatten (Flatten) (None, 512) 0
dense (Dense) (None, 256) 131328
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 14,887,684
Trainable params: 7,252,292
Non-trainable params: 7,635,392
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 60s 123ms/step - loss: 1.1853 - accuracy: 0.4585 - val_loss: 0.9170 - val_accuracy: 0.5789 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 58s 122ms/step - loss: 1.0514 - accuracy: 0.5244 - val_loss: 0.9705 - val_accuracy: 0.5971 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 58s 122ms/step - loss: 0.9214 - accuracy: 0.5940 - val_loss: 0.8417 - val_accuracy: 0.6438 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 56s 118ms/step - loss: 0.8536 - accuracy: 0.6485 - val_loss: 0.9996 - val_accuracy: 0.5863 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 57s 121ms/step - loss: 0.8037 - accuracy: 0.6740 - val_loss: 0.8071 - val_accuracy: 0.6713 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 57s 121ms/step - loss: 0.7712 - accuracy: 0.6879 - val_loss: 0.7755 - val_accuracy: 0.6769 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 58s 122ms/step - loss: 0.7429 - accuracy: 0.7063 - val_loss: 0.7747 - val_accuracy: 0.6799 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 56s 119ms/step - loss: 0.7550 - accuracy: 0.6960 - val_loss: 0.9831 - val_accuracy: 0.5990 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 56s 118ms/step - loss: 0.7480 - accuracy: 0.7009 - val_loss: 1.0221 - val_accuracy: 0.5720 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 57s 120ms/step - loss: 0.7018 - accuracy: 0.7259 - val_loss: 0.7804 - val_accuracy: 0.6888 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 57s 121ms/step - loss: 0.6038 - accuracy: 0.7641 - val_loss: 0.7710 - val_accuracy: 0.7022 - lr: 2.0000e-04
Epoch 12/20
473/473 [==============================] - 58s 122ms/step - loss: 0.5686 - accuracy: 0.7844 - val_loss: 0.7590 - val_accuracy: 0.7129 - lr: 2.0000e-04
Epoch 13/20
473/473 [==============================] - 56s 119ms/step - loss: 0.5485 - accuracy: 0.7941 - val_loss: 0.7905 - val_accuracy: 0.6954 - lr: 2.0000e-04
Epoch 14/20
473/473 [==============================] - 56s 119ms/step - loss: 0.5202 - accuracy: 0.8088 - val_loss: 0.8029 - val_accuracy: 0.7022 - lr: 2.0000e-04
Epoch 15/20
473/473 [==============================] - 56s 118ms/step - loss: 0.5028 - accuracy: 0.8125 - val_loss: 0.8014 - val_accuracy: 0.7026 - lr: 2.0000e-04
Epoch 16/20
473/473 [==============================] - 56s 119ms/step - loss: 0.4603 - accuracy: 0.8345 - val_loss: 0.8063 - val_accuracy: 0.7016 - lr: 4.0000e-05
Epoch 17/20
473/473 [==============================] - 56s 119ms/step - loss: 0.4422 - accuracy: 0.8435 - val_loss: 0.8098 - val_accuracy: 0.7048 - lr: 4.0000e-05
1/1 [==============================] - 0s 78ms/step - loss: 0.5211 - accuracy: 0.8125
1/1 [==============================] - 0s 240ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 1 3 1 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 0 1 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 0.90 0.90 0.90 10
neutral 0.50 1.00 0.67 5
sad 1.00 0.60 0.75 10
surprise 1.00 0.86 0.92 7
accuracy 0.81 32
macro avg 0.85 0.84 0.81 32
weighted avg 0.89 0.81 0.82 32
model_name = 'Model 3: VGG16'
save_object(history_vgg16, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_vgg16, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_vgg16, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_vgg16, data_type='Training Set')
loss_and_accuracy_comparisson(history_vgg16, data_type='Validation Set')
metric_vgg16.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 3: VGG16 (32) - RGB | 0.651267 | 0.573237 | 0.625000 |
| Model 3: Tuned VGG16 (32) - RGB | 0.784367 | 0.712879 | 0.812500 |
Think About It:
Note: You can even go back and build your own architecture on top of the VGG16 Transfer layer and see if you can improve the performance
⏩ Observations and Insights:
⏩ Recommendations:
ap.ResNet101(include_top=False, weights="imagenet", input_shape=(48, 48, 3)).summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet101_weights_tf_dim_ordering_tf_kernels_notop.h5
171446536/171446536 [==============================] - 1s 0us/step
Model: "resnet101"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 48, 48, 3)] 0 []
conv1_pad (ZeroPadding2D) (None, 54, 54, 3) 0 ['input_2[0][0]']
conv1_conv (Conv2D) (None, 24, 24, 64) 9472 ['conv1_pad[0][0]']
conv1_bn (BatchNormalization) (None, 24, 24, 64) 256 ['conv1_conv[0][0]']
conv1_relu (Activation) (None, 24, 24, 64) 0 ['conv1_bn[0][0]']
pool1_pad (ZeroPadding2D) (None, 26, 26, 64) 0 ['conv1_relu[0][0]']
pool1_pool (MaxPooling2D) (None, 12, 12, 64) 0 ['pool1_pad[0][0]']
conv2_block1_1_conv (Conv2D) (None, 12, 12, 64) 4160 ['pool1_pool[0][0]']
conv2_block1_1_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block1_1_conv[0][0]']
ization)
conv2_block1_1_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block1_1_bn[0][0]']
n)
conv2_block1_2_conv (Conv2D) (None, 12, 12, 64) 36928 ['conv2_block1_1_relu[0][0]']
conv2_block1_2_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block1_2_conv[0][0]']
ization)
conv2_block1_2_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block1_2_bn[0][0]']
n)
conv2_block1_0_conv (Conv2D) (None, 12, 12, 256) 16640 ['pool1_pool[0][0]']
conv2_block1_3_conv (Conv2D) (None, 12, 12, 256) 16640 ['conv2_block1_2_relu[0][0]']
conv2_block1_0_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block1_0_conv[0][0]']
ization)
conv2_block1_3_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block1_3_conv[0][0]']
ization)
conv2_block1_add (Add) (None, 12, 12, 256) 0 ['conv2_block1_0_bn[0][0]',
'conv2_block1_3_bn[0][0]']
conv2_block1_out (Activation) (None, 12, 12, 256) 0 ['conv2_block1_add[0][0]']
conv2_block2_1_conv (Conv2D) (None, 12, 12, 64) 16448 ['conv2_block1_out[0][0]']
conv2_block2_1_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block2_1_conv[0][0]']
ization)
conv2_block2_1_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block2_1_bn[0][0]']
n)
conv2_block2_2_conv (Conv2D) (None, 12, 12, 64) 36928 ['conv2_block2_1_relu[0][0]']
conv2_block2_2_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block2_2_conv[0][0]']
ization)
conv2_block2_2_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block2_2_bn[0][0]']
n)
conv2_block2_3_conv (Conv2D) (None, 12, 12, 256) 16640 ['conv2_block2_2_relu[0][0]']
conv2_block2_3_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block2_3_conv[0][0]']
ization)
conv2_block2_add (Add) (None, 12, 12, 256) 0 ['conv2_block1_out[0][0]',
'conv2_block2_3_bn[0][0]']
conv2_block2_out (Activation) (None, 12, 12, 256) 0 ['conv2_block2_add[0][0]']
conv2_block3_1_conv (Conv2D) (None, 12, 12, 64) 16448 ['conv2_block2_out[0][0]']
conv2_block3_1_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block3_1_conv[0][0]']
ization)
conv2_block3_1_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block3_1_bn[0][0]']
n)
conv2_block3_2_conv (Conv2D) (None, 12, 12, 64) 36928 ['conv2_block3_1_relu[0][0]']
conv2_block3_2_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block3_2_conv[0][0]']
ization)
conv2_block3_2_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block3_2_bn[0][0]']
n)
conv2_block3_3_conv (Conv2D) (None, 12, 12, 256) 16640 ['conv2_block3_2_relu[0][0]']
conv2_block3_3_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block3_3_conv[0][0]']
ization)
conv2_block3_add (Add) (None, 12, 12, 256) 0 ['conv2_block2_out[0][0]',
'conv2_block3_3_bn[0][0]']
conv2_block3_out (Activation) (None, 12, 12, 256) 0 ['conv2_block3_add[0][0]']
conv3_block1_1_conv (Conv2D) (None, 6, 6, 128) 32896 ['conv2_block3_out[0][0]']
conv3_block1_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block1_1_conv[0][0]']
ization)
conv3_block1_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block1_1_bn[0][0]']
n)
conv3_block1_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block1_1_relu[0][0]']
conv3_block1_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block1_2_conv[0][0]']
ization)
conv3_block1_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block1_2_bn[0][0]']
n)
conv3_block1_0_conv (Conv2D) (None, 6, 6, 512) 131584 ['conv2_block3_out[0][0]']
conv3_block1_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block1_2_relu[0][0]']
conv3_block1_0_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block1_0_conv[0][0]']
ization)
conv3_block1_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block1_3_conv[0][0]']
ization)
conv3_block1_add (Add) (None, 6, 6, 512) 0 ['conv3_block1_0_bn[0][0]',
'conv3_block1_3_bn[0][0]']
conv3_block1_out (Activation) (None, 6, 6, 512) 0 ['conv3_block1_add[0][0]']
conv3_block2_1_conv (Conv2D) (None, 6, 6, 128) 65664 ['conv3_block1_out[0][0]']
conv3_block2_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block2_1_conv[0][0]']
ization)
conv3_block2_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block2_1_bn[0][0]']
n)
conv3_block2_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block2_1_relu[0][0]']
conv3_block2_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block2_2_conv[0][0]']
ization)
conv3_block2_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block2_2_bn[0][0]']
n)
conv3_block2_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block2_2_relu[0][0]']
conv3_block2_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block2_3_conv[0][0]']
ization)
conv3_block2_add (Add) (None, 6, 6, 512) 0 ['conv3_block1_out[0][0]',
'conv3_block2_3_bn[0][0]']
conv3_block2_out (Activation) (None, 6, 6, 512) 0 ['conv3_block2_add[0][0]']
conv3_block3_1_conv (Conv2D) (None, 6, 6, 128) 65664 ['conv3_block2_out[0][0]']
conv3_block3_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block3_1_conv[0][0]']
ization)
conv3_block3_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block3_1_bn[0][0]']
n)
conv3_block3_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block3_1_relu[0][0]']
conv3_block3_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block3_2_conv[0][0]']
ization)
conv3_block3_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block3_2_bn[0][0]']
n)
conv3_block3_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block3_2_relu[0][0]']
conv3_block3_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block3_3_conv[0][0]']
ization)
conv3_block3_add (Add) (None, 6, 6, 512) 0 ['conv3_block2_out[0][0]',
'conv3_block3_3_bn[0][0]']
conv3_block3_out (Activation) (None, 6, 6, 512) 0 ['conv3_block3_add[0][0]']
conv3_block4_1_conv (Conv2D) (None, 6, 6, 128) 65664 ['conv3_block3_out[0][0]']
conv3_block4_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block4_1_conv[0][0]']
ization)
conv3_block4_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block4_1_bn[0][0]']
n)
conv3_block4_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block4_1_relu[0][0]']
conv3_block4_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block4_2_conv[0][0]']
ization)
conv3_block4_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block4_2_bn[0][0]']
n)
conv3_block4_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block4_2_relu[0][0]']
conv3_block4_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block4_3_conv[0][0]']
ization)
conv3_block4_add (Add) (None, 6, 6, 512) 0 ['conv3_block3_out[0][0]',
'conv3_block4_3_bn[0][0]']
conv3_block4_out (Activation) (None, 6, 6, 512) 0 ['conv3_block4_add[0][0]']
conv4_block1_1_conv (Conv2D) (None, 3, 3, 256) 131328 ['conv3_block4_out[0][0]']
conv4_block1_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block1_1_conv[0][0]']
ization)
conv4_block1_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block1_1_bn[0][0]']
n)
conv4_block1_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block1_1_relu[0][0]']
conv4_block1_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block1_2_conv[0][0]']
ization)
conv4_block1_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block1_2_bn[0][0]']
n)
conv4_block1_0_conv (Conv2D) (None, 3, 3, 1024) 525312 ['conv3_block4_out[0][0]']
conv4_block1_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block1_2_relu[0][0]']
conv4_block1_0_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block1_0_conv[0][0]']
ization)
conv4_block1_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block1_3_conv[0][0]']
ization)
conv4_block1_add (Add) (None, 3, 3, 1024) 0 ['conv4_block1_0_bn[0][0]',
'conv4_block1_3_bn[0][0]']
conv4_block1_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block1_add[0][0]']
conv4_block2_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block1_out[0][0]']
conv4_block2_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block2_1_conv[0][0]']
ization)
conv4_block2_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block2_1_bn[0][0]']
n)
conv4_block2_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block2_1_relu[0][0]']
conv4_block2_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block2_2_conv[0][0]']
ization)
conv4_block2_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block2_2_bn[0][0]']
n)
conv4_block2_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block2_2_relu[0][0]']
conv4_block2_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block2_3_conv[0][0]']
ization)
conv4_block2_add (Add) (None, 3, 3, 1024) 0 ['conv4_block1_out[0][0]',
'conv4_block2_3_bn[0][0]']
conv4_block2_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block2_add[0][0]']
conv4_block3_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block2_out[0][0]']
conv4_block3_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block3_1_conv[0][0]']
ization)
conv4_block3_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block3_1_bn[0][0]']
n)
conv4_block3_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block3_1_relu[0][0]']
conv4_block3_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block3_2_conv[0][0]']
ization)
conv4_block3_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block3_2_bn[0][0]']
n)
conv4_block3_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block3_2_relu[0][0]']
conv4_block3_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block3_3_conv[0][0]']
ization)
conv4_block3_add (Add) (None, 3, 3, 1024) 0 ['conv4_block2_out[0][0]',
'conv4_block3_3_bn[0][0]']
conv4_block3_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block3_add[0][0]']
conv4_block4_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block3_out[0][0]']
conv4_block4_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block4_1_conv[0][0]']
ization)
conv4_block4_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block4_1_bn[0][0]']
n)
conv4_block4_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block4_1_relu[0][0]']
conv4_block4_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block4_2_conv[0][0]']
ization)
conv4_block4_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block4_2_bn[0][0]']
n)
conv4_block4_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block4_2_relu[0][0]']
conv4_block4_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block4_3_conv[0][0]']
ization)
conv4_block4_add (Add) (None, 3, 3, 1024) 0 ['conv4_block3_out[0][0]',
'conv4_block4_3_bn[0][0]']
conv4_block4_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block4_add[0][0]']
conv4_block5_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block4_out[0][0]']
conv4_block5_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block5_1_conv[0][0]']
ization)
conv4_block5_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block5_1_bn[0][0]']
n)
conv4_block5_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block5_1_relu[0][0]']
conv4_block5_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block5_2_conv[0][0]']
ization)
conv4_block5_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block5_2_bn[0][0]']
n)
conv4_block5_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block5_2_relu[0][0]']
conv4_block5_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block5_3_conv[0][0]']
ization)
conv4_block5_add (Add) (None, 3, 3, 1024) 0 ['conv4_block4_out[0][0]',
'conv4_block5_3_bn[0][0]']
conv4_block5_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block5_add[0][0]']
conv4_block6_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block5_out[0][0]']
conv4_block6_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block6_1_conv[0][0]']
ization)
conv4_block6_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block6_1_bn[0][0]']
n)
conv4_block6_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block6_1_relu[0][0]']
conv4_block6_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block6_2_conv[0][0]']
ization)
conv4_block6_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block6_2_bn[0][0]']
n)
conv4_block6_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block6_2_relu[0][0]']
conv4_block6_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block6_3_conv[0][0]']
ization)
conv4_block6_add (Add) (None, 3, 3, 1024) 0 ['conv4_block5_out[0][0]',
'conv4_block6_3_bn[0][0]']
conv4_block6_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block6_add[0][0]']
conv4_block7_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block6_out[0][0]']
conv4_block7_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block7_1_conv[0][0]']
ization)
conv4_block7_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block7_1_bn[0][0]']
n)
conv4_block7_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block7_1_relu[0][0]']
conv4_block7_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block7_2_conv[0][0]']
ization)
conv4_block7_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block7_2_bn[0][0]']
n)
conv4_block7_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block7_2_relu[0][0]']
conv4_block7_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block7_3_conv[0][0]']
ization)
conv4_block7_add (Add) (None, 3, 3, 1024) 0 ['conv4_block6_out[0][0]',
'conv4_block7_3_bn[0][0]']
conv4_block7_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block7_add[0][0]']
conv4_block8_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block7_out[0][0]']
conv4_block8_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block8_1_conv[0][0]']
ization)
conv4_block8_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block8_1_bn[0][0]']
n)
conv4_block8_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block8_1_relu[0][0]']
conv4_block8_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block8_2_conv[0][0]']
ization)
conv4_block8_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block8_2_bn[0][0]']
n)
conv4_block8_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block8_2_relu[0][0]']
conv4_block8_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block8_3_conv[0][0]']
ization)
conv4_block8_add (Add) (None, 3, 3, 1024) 0 ['conv4_block7_out[0][0]',
'conv4_block8_3_bn[0][0]']
conv4_block8_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block8_add[0][0]']
conv4_block9_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block8_out[0][0]']
conv4_block9_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block9_1_conv[0][0]']
ization)
conv4_block9_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block9_1_bn[0][0]']
n)
conv4_block9_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block9_1_relu[0][0]']
conv4_block9_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block9_2_conv[0][0]']
ization)
conv4_block9_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block9_2_bn[0][0]']
n)
conv4_block9_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block9_2_relu[0][0]']
conv4_block9_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block9_3_conv[0][0]']
ization)
conv4_block9_add (Add) (None, 3, 3, 1024) 0 ['conv4_block8_out[0][0]',
'conv4_block9_3_bn[0][0]']
conv4_block9_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block9_add[0][0]']
conv4_block10_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block9_out[0][0]']
conv4_block10_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block10_1_conv[0][0]']
lization)
conv4_block10_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block10_1_bn[0][0]']
on)
conv4_block10_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block10_1_relu[0][0]']
conv4_block10_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block10_2_conv[0][0]']
lization)
conv4_block10_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block10_2_bn[0][0]']
on)
conv4_block10_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block10_2_relu[0][0]']
conv4_block10_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block10_3_conv[0][0]']
lization)
conv4_block10_add (Add) (None, 3, 3, 1024) 0 ['conv4_block9_out[0][0]',
'conv4_block10_3_bn[0][0]']
conv4_block10_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block10_add[0][0]']
conv4_block11_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block10_out[0][0]']
conv4_block11_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block11_1_conv[0][0]']
lization)
conv4_block11_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block11_1_bn[0][0]']
on)
conv4_block11_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block11_1_relu[0][0]']
conv4_block11_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block11_2_conv[0][0]']
lization)
conv4_block11_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block11_2_bn[0][0]']
on)
conv4_block11_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block11_2_relu[0][0]']
conv4_block11_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block11_3_conv[0][0]']
lization)
conv4_block11_add (Add) (None, 3, 3, 1024) 0 ['conv4_block10_out[0][0]',
'conv4_block11_3_bn[0][0]']
conv4_block11_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block11_add[0][0]']
conv4_block12_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block11_out[0][0]']
conv4_block12_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block12_1_conv[0][0]']
lization)
conv4_block12_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block12_1_bn[0][0]']
on)
conv4_block12_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block12_1_relu[0][0]']
conv4_block12_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block12_2_conv[0][0]']
lization)
conv4_block12_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block12_2_bn[0][0]']
on)
conv4_block12_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block12_2_relu[0][0]']
conv4_block12_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block12_3_conv[0][0]']
lization)
conv4_block12_add (Add) (None, 3, 3, 1024) 0 ['conv4_block11_out[0][0]',
'conv4_block12_3_bn[0][0]']
conv4_block12_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block12_add[0][0]']
conv4_block13_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block12_out[0][0]']
conv4_block13_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block13_1_conv[0][0]']
lization)
conv4_block13_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block13_1_bn[0][0]']
on)
conv4_block13_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block13_1_relu[0][0]']
conv4_block13_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block13_2_conv[0][0]']
lization)
conv4_block13_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block13_2_bn[0][0]']
on)
conv4_block13_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block13_2_relu[0][0]']
conv4_block13_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block13_3_conv[0][0]']
lization)
conv4_block13_add (Add) (None, 3, 3, 1024) 0 ['conv4_block12_out[0][0]',
'conv4_block13_3_bn[0][0]']
conv4_block13_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block13_add[0][0]']
conv4_block14_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block13_out[0][0]']
conv4_block14_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block14_1_conv[0][0]']
lization)
conv4_block14_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block14_1_bn[0][0]']
on)
conv4_block14_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block14_1_relu[0][0]']
conv4_block14_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block14_2_conv[0][0]']
lization)
conv4_block14_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block14_2_bn[0][0]']
on)
conv4_block14_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block14_2_relu[0][0]']
conv4_block14_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block14_3_conv[0][0]']
lization)
conv4_block14_add (Add) (None, 3, 3, 1024) 0 ['conv4_block13_out[0][0]',
'conv4_block14_3_bn[0][0]']
conv4_block14_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block14_add[0][0]']
conv4_block15_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block14_out[0][0]']
conv4_block15_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block15_1_conv[0][0]']
lization)
conv4_block15_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block15_1_bn[0][0]']
on)
conv4_block15_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block15_1_relu[0][0]']
conv4_block15_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block15_2_conv[0][0]']
lization)
conv4_block15_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block15_2_bn[0][0]']
on)
conv4_block15_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block15_2_relu[0][0]']
conv4_block15_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block15_3_conv[0][0]']
lization)
conv4_block15_add (Add) (None, 3, 3, 1024) 0 ['conv4_block14_out[0][0]',
'conv4_block15_3_bn[0][0]']
conv4_block15_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block15_add[0][0]']
conv4_block16_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block15_out[0][0]']
conv4_block16_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block16_1_conv[0][0]']
lization)
conv4_block16_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block16_1_bn[0][0]']
on)
conv4_block16_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block16_1_relu[0][0]']
conv4_block16_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block16_2_conv[0][0]']
lization)
conv4_block16_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block16_2_bn[0][0]']
on)
conv4_block16_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block16_2_relu[0][0]']
conv4_block16_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block16_3_conv[0][0]']
lization)
conv4_block16_add (Add) (None, 3, 3, 1024) 0 ['conv4_block15_out[0][0]',
'conv4_block16_3_bn[0][0]']
conv4_block16_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block16_add[0][0]']
conv4_block17_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block16_out[0][0]']
conv4_block17_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block17_1_conv[0][0]']
lization)
conv4_block17_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block17_1_bn[0][0]']
on)
conv4_block17_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block17_1_relu[0][0]']
conv4_block17_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block17_2_conv[0][0]']
lization)
conv4_block17_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block17_2_bn[0][0]']
on)
conv4_block17_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block17_2_relu[0][0]']
conv4_block17_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block17_3_conv[0][0]']
lization)
conv4_block17_add (Add) (None, 3, 3, 1024) 0 ['conv4_block16_out[0][0]',
'conv4_block17_3_bn[0][0]']
conv4_block17_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block17_add[0][0]']
conv4_block18_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block17_out[0][0]']
conv4_block18_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block18_1_conv[0][0]']
lization)
conv4_block18_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block18_1_bn[0][0]']
on)
conv4_block18_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block18_1_relu[0][0]']
conv4_block18_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block18_2_conv[0][0]']
lization)
conv4_block18_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block18_2_bn[0][0]']
on)
conv4_block18_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block18_2_relu[0][0]']
conv4_block18_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block18_3_conv[0][0]']
lization)
conv4_block18_add (Add) (None, 3, 3, 1024) 0 ['conv4_block17_out[0][0]',
'conv4_block18_3_bn[0][0]']
conv4_block18_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block18_add[0][0]']
conv4_block19_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block18_out[0][0]']
conv4_block19_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block19_1_conv[0][0]']
lization)
conv4_block19_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block19_1_bn[0][0]']
on)
conv4_block19_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block19_1_relu[0][0]']
conv4_block19_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block19_2_conv[0][0]']
lization)
conv4_block19_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block19_2_bn[0][0]']
on)
conv4_block19_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block19_2_relu[0][0]']
conv4_block19_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block19_3_conv[0][0]']
lization)
conv4_block19_add (Add) (None, 3, 3, 1024) 0 ['conv4_block18_out[0][0]',
'conv4_block19_3_bn[0][0]']
conv4_block19_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block19_add[0][0]']
conv4_block20_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block19_out[0][0]']
conv4_block20_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block20_1_conv[0][0]']
lization)
conv4_block20_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block20_1_bn[0][0]']
on)
conv4_block20_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block20_1_relu[0][0]']
conv4_block20_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block20_2_conv[0][0]']
lization)
conv4_block20_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block20_2_bn[0][0]']
on)
conv4_block20_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block20_2_relu[0][0]']
conv4_block20_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block20_3_conv[0][0]']
lization)
conv4_block20_add (Add) (None, 3, 3, 1024) 0 ['conv4_block19_out[0][0]',
'conv4_block20_3_bn[0][0]']
conv4_block20_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block20_add[0][0]']
conv4_block21_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block20_out[0][0]']
conv4_block21_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block21_1_conv[0][0]']
lization)
conv4_block21_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block21_1_bn[0][0]']
on)
conv4_block21_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block21_1_relu[0][0]']
conv4_block21_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block21_2_conv[0][0]']
lization)
conv4_block21_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block21_2_bn[0][0]']
on)
conv4_block21_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block21_2_relu[0][0]']
conv4_block21_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block21_3_conv[0][0]']
lization)
conv4_block21_add (Add) (None, 3, 3, 1024) 0 ['conv4_block20_out[0][0]',
'conv4_block21_3_bn[0][0]']
conv4_block21_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block21_add[0][0]']
conv4_block22_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block21_out[0][0]']
conv4_block22_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block22_1_conv[0][0]']
lization)
conv4_block22_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block22_1_bn[0][0]']
on)
conv4_block22_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block22_1_relu[0][0]']
conv4_block22_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block22_2_conv[0][0]']
lization)
conv4_block22_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block22_2_bn[0][0]']
on)
conv4_block22_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block22_2_relu[0][0]']
conv4_block22_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block22_3_conv[0][0]']
lization)
conv4_block22_add (Add) (None, 3, 3, 1024) 0 ['conv4_block21_out[0][0]',
'conv4_block22_3_bn[0][0]']
conv4_block22_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block22_add[0][0]']
conv4_block23_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block22_out[0][0]']
conv4_block23_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block23_1_conv[0][0]']
lization)
conv4_block23_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block23_1_bn[0][0]']
on)
conv4_block23_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block23_1_relu[0][0]']
conv4_block23_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block23_2_conv[0][0]']
lization)
conv4_block23_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block23_2_bn[0][0]']
on)
conv4_block23_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block23_2_relu[0][0]']
conv4_block23_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block23_3_conv[0][0]']
lization)
conv4_block23_add (Add) (None, 3, 3, 1024) 0 ['conv4_block22_out[0][0]',
'conv4_block23_3_bn[0][0]']
conv4_block23_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block23_add[0][0]']
conv5_block1_1_conv (Conv2D) (None, 2, 2, 512) 524800 ['conv4_block23_out[0][0]']
conv5_block1_1_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block1_1_conv[0][0]']
ization)
conv5_block1_1_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block1_1_bn[0][0]']
n)
conv5_block1_2_conv (Conv2D) (None, 2, 2, 512) 2359808 ['conv5_block1_1_relu[0][0]']
conv5_block1_2_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block1_2_conv[0][0]']
ization)
conv5_block1_2_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block1_2_bn[0][0]']
n)
conv5_block1_0_conv (Conv2D) (None, 2, 2, 2048) 2099200 ['conv4_block23_out[0][0]']
conv5_block1_3_conv (Conv2D) (None, 2, 2, 2048) 1050624 ['conv5_block1_2_relu[0][0]']
conv5_block1_0_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block1_0_conv[0][0]']
ization)
conv5_block1_3_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block1_3_conv[0][0]']
ization)
conv5_block1_add (Add) (None, 2, 2, 2048) 0 ['conv5_block1_0_bn[0][0]',
'conv5_block1_3_bn[0][0]']
conv5_block1_out (Activation) (None, 2, 2, 2048) 0 ['conv5_block1_add[0][0]']
conv5_block2_1_conv (Conv2D) (None, 2, 2, 512) 1049088 ['conv5_block1_out[0][0]']
conv5_block2_1_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block2_1_conv[0][0]']
ization)
conv5_block2_1_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block2_1_bn[0][0]']
n)
conv5_block2_2_conv (Conv2D) (None, 2, 2, 512) 2359808 ['conv5_block2_1_relu[0][0]']
conv5_block2_2_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block2_2_conv[0][0]']
ization)
conv5_block2_2_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block2_2_bn[0][0]']
n)
conv5_block2_3_conv (Conv2D) (None, 2, 2, 2048) 1050624 ['conv5_block2_2_relu[0][0]']
conv5_block2_3_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block2_3_conv[0][0]']
ization)
conv5_block2_add (Add) (None, 2, 2, 2048) 0 ['conv5_block1_out[0][0]',
'conv5_block2_3_bn[0][0]']
conv5_block2_out (Activation) (None, 2, 2, 2048) 0 ['conv5_block2_add[0][0]']
conv5_block3_1_conv (Conv2D) (None, 2, 2, 512) 1049088 ['conv5_block2_out[0][0]']
conv5_block3_1_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block3_1_conv[0][0]']
ization)
conv5_block3_1_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block3_1_bn[0][0]']
n)
conv5_block3_2_conv (Conv2D) (None, 2, 2, 512) 2359808 ['conv5_block3_1_relu[0][0]']
conv5_block3_2_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block3_2_conv[0][0]']
ization)
conv5_block3_2_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block3_2_bn[0][0]']
n)
conv5_block3_3_conv (Conv2D) (None, 2, 2, 2048) 1050624 ['conv5_block3_2_relu[0][0]']
conv5_block3_3_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block3_3_conv[0][0]']
ization)
conv5_block3_add (Add) (None, 2, 2, 2048) 0 ['conv5_block2_out[0][0]',
'conv5_block3_3_bn[0][0]']
conv5_block3_out (Activation) (None, 2, 2, 2048) 0 ['conv5_block3_add[0][0]']
==================================================================================================
Total params: 42,658,176
Trainable params: 42,552,832
Non-trainable params: 105,344
__________________________________________________________________________________________________
# Model 4: ResNet V2
def model_builder_resnet(input_shape, tune=0):
"""Transfer approach with the ResNet V2
"""
resnet_model = ap.ResNet101(include_top=False, weights="imagenet", input_shape=input_shape)
# Making all the layers of the VGG model non-trainable
if tune > 0:
for layer in resnet_model.layers[:-tune]:
layer.trainable = False
else:
for layer in resnet_model.layers:
layer.trainable = False
model = Sequential([
resnet_model,
Flatten(),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dropout(0.3),
Dense(64, activation='relu'),
BatchNormalization(),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_resnet, metric_resnet, history_resnet = model_building_and_evaluating_process(
model_builder_resnet, f'Model 4: Resnet V2 ({batch_size})', batch_size=batch_size, epochs=20, include_grayscale=False
)
---------------------------------------------
Model 4: Resnet V2 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
resnet101 (Functional) (None, 2, 2, 2048) 42658176
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 256) 2097408
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 44,797,252
Trainable params: 2,138,948
Non-trainable params: 42,658,304
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 73s 140ms/step - loss: 1.4625 - accuracy: 0.2616 - val_loss: 1.3638 - val_accuracy: 0.2447 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 62s 131ms/step - loss: 1.3958 - accuracy: 0.2665 - val_loss: 1.3371 - val_accuracy: 0.3378 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 58s 123ms/step - loss: 1.3655 - accuracy: 0.3060 - val_loss: 1.5133 - val_accuracy: 0.1836 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 58s 122ms/step - loss: 1.3194 - accuracy: 0.3426 - val_loss: 1.5810 - val_accuracy: 0.1943 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 59s 124ms/step - loss: 1.2924 - accuracy: 0.3683 - val_loss: 1.5175 - val_accuracy: 0.2504 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 60s 127ms/step - loss: 1.2399 - accuracy: 0.4201 - val_loss: 1.4819 - val_accuracy: 0.3183 - lr: 2.0000e-04
Epoch 7/20
473/473 [==============================] - 61s 129ms/step - loss: 1.2306 - accuracy: 0.4280 - val_loss: 1.2199 - val_accuracy: 0.4354 - lr: 2.0000e-04
Epoch 8/20
473/473 [==============================] - 60s 128ms/step - loss: 1.2215 - accuracy: 0.4339 - val_loss: 1.2098 - val_accuracy: 0.4493 - lr: 2.0000e-04
Epoch 9/20
473/473 [==============================] - 60s 127ms/step - loss: 1.2194 - accuracy: 0.4398 - val_loss: 1.2113 - val_accuracy: 0.4517 - lr: 2.0000e-04
Epoch 10/20
473/473 [==============================] - 58s 123ms/step - loss: 1.2133 - accuracy: 0.4459 - val_loss: 1.3069 - val_accuracy: 0.3908 - lr: 2.0000e-04
Epoch 11/20
473/473 [==============================] - 61s 128ms/step - loss: 1.2091 - accuracy: 0.4492 - val_loss: 1.1824 - val_accuracy: 0.4678 - lr: 2.0000e-04
Epoch 12/20
473/473 [==============================] - 59s 125ms/step - loss: 1.2051 - accuracy: 0.4526 - val_loss: 1.1814 - val_accuracy: 0.4671 - lr: 2.0000e-04
Epoch 13/20
473/473 [==============================] - 61s 128ms/step - loss: 1.2002 - accuracy: 0.4555 - val_loss: 1.1783 - val_accuracy: 0.4778 - lr: 2.0000e-04
Epoch 14/20
473/473 [==============================] - 58s 122ms/step - loss: 1.1955 - accuracy: 0.4554 - val_loss: 1.1905 - val_accuracy: 0.4669 - lr: 2.0000e-04
Epoch 15/20
473/473 [==============================] - 58s 122ms/step - loss: 1.1942 - accuracy: 0.4567 - val_loss: 1.3811 - val_accuracy: 0.3641 - lr: 2.0000e-04
Epoch 16/20
473/473 [==============================] - 58s 123ms/step - loss: 1.1909 - accuracy: 0.4618 - val_loss: 1.1910 - val_accuracy: 0.4736 - lr: 2.0000e-04
Epoch 17/20
473/473 [==============================] - 58s 124ms/step - loss: 1.1796 - accuracy: 0.4757 - val_loss: 1.1884 - val_accuracy: 0.4643 - lr: 4.0000e-05
Epoch 18/20
473/473 [==============================] - 58s 122ms/step - loss: 1.1780 - accuracy: 0.4677 - val_loss: 1.2073 - val_accuracy: 0.4541 - lr: 4.0000e-05
1/1 [==============================] - 0s 111ms/step - loss: 1.1514 - accuracy: 0.5312
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 2 1 3 2 1 0 2 2 2 2 2 0 0 3 1 0 0 2 2 1 1 0 2 0 0 2 0 0 0]
precision recall f1-score support
happy 0.58 0.70 0.64 10
neutral 0.50 0.60 0.55 5
sad 0.36 0.40 0.38 10
surprise 1.00 0.43 0.60 7
accuracy 0.53 32
macro avg 0.61 0.53 0.54 32
weighted avg 0.59 0.53 0.53 32
# Let's try again tuning some of the last layers in the transfer model
batch_size = 32
model, results, history = model_building_and_evaluating_process(model_builder_resnet, f'Model 4: Tuned Resnet V2 ({batch_size})',
batch_size=batch_size, epochs=20, tune=5, include_grayscale=False)
model_resnet.update(model)
metric_resnet = pd.concat([metric_resnet, results])
history_resnet.update(history)
---------------------------------------------
Model 4: Tuned Resnet V2 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
resnet101 (Functional) (None, 2, 2, 2048) 42658176
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 256) 2097408
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 44,797,252
Trainable params: 3,193,668
Non-trainable params: 41,603,584
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 72s 136ms/step - loss: 1.4439 - accuracy: 0.2798 - val_loss: 1.3643 - val_accuracy: 0.2339 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 62s 131ms/step - loss: 1.3400 - accuracy: 0.3270 - val_loss: 1.4209 - val_accuracy: 0.2522 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 62s 131ms/step - loss: 1.3090 - accuracy: 0.3579 - val_loss: 1.3305 - val_accuracy: 0.3390 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 60s 126ms/step - loss: 1.2867 - accuracy: 0.3843 - val_loss: 1.4322 - val_accuracy: 0.2518 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 59s 125ms/step - loss: 1.2718 - accuracy: 0.3998 - val_loss: 1.3774 - val_accuracy: 0.3138 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 62s 132ms/step - loss: 1.2544 - accuracy: 0.4141 - val_loss: 1.3111 - val_accuracy: 0.3659 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 62s 132ms/step - loss: 1.2385 - accuracy: 0.4211 - val_loss: 1.2145 - val_accuracy: 0.4400 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 61s 128ms/step - loss: 1.2297 - accuracy: 0.4289 - val_loss: 1.3236 - val_accuracy: 0.3500 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 60s 127ms/step - loss: 1.2218 - accuracy: 0.4342 - val_loss: 1.5070 - val_accuracy: 0.3516 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 62s 132ms/step - loss: 1.2037 - accuracy: 0.4498 - val_loss: 1.1793 - val_accuracy: 0.4601 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 62s 131ms/step - loss: 1.1928 - accuracy: 0.4614 - val_loss: 1.2029 - val_accuracy: 0.4641 - lr: 0.0010
Epoch 12/20
473/473 [==============================] - 60s 128ms/step - loss: 1.1891 - accuracy: 0.4591 - val_loss: 1.4216 - val_accuracy: 0.4089 - lr: 0.0010
Epoch 13/20
473/473 [==============================] - 60s 127ms/step - loss: 1.1772 - accuracy: 0.4688 - val_loss: 1.4118 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 14/20
473/473 [==============================] - 62s 131ms/step - loss: 1.1333 - accuracy: 0.5008 - val_loss: 1.1229 - val_accuracy: 0.4967 - lr: 2.0000e-04
Epoch 15/20
473/473 [==============================] - 62s 131ms/step - loss: 1.1315 - accuracy: 0.5072 - val_loss: 1.1263 - val_accuracy: 0.5011 - lr: 2.0000e-04
Epoch 16/20
473/473 [==============================] - 60s 127ms/step - loss: 1.1244 - accuracy: 0.5091 - val_loss: 1.1475 - val_accuracy: 0.4895 - lr: 2.0000e-04
Epoch 17/20
473/473 [==============================] - 62s 130ms/step - loss: 1.1192 - accuracy: 0.5137 - val_loss: 1.1321 - val_accuracy: 0.5067 - lr: 2.0000e-04
Epoch 18/20
473/473 [==============================] - 60s 127ms/step - loss: 1.1115 - accuracy: 0.5130 - val_loss: 1.1718 - val_accuracy: 0.4657 - lr: 4.0000e-05
Epoch 19/20
473/473 [==============================] - 59s 125ms/step - loss: 1.1112 - accuracy: 0.5166 - val_loss: 1.1363 - val_accuracy: 0.4999 - lr: 4.0000e-05
1/1 [==============================] - 0s 113ms/step - loss: 1.1130 - accuracy: 0.3750
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 0 3 1 1 0 1 1 0 3 2 2 2 0 1 0 3 1 0 0 0 2 1 1 0 2 0 1 1 0 0 0]
precision recall f1-score support
happy 0.43 0.60 0.50 10
neutral 0.30 0.60 0.40 5
sad 0.20 0.10 0.13 10
surprise 0.67 0.29 0.40 7
accuracy 0.38 32
macro avg 0.40 0.40 0.36 32
weighted avg 0.39 0.38 0.35 32
model_name = 'Model 4: Resnet V2'
save_object(history_resnet, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_resnet, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_resnet, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_resnet, data_type='Training Set')
loss_and_accuracy_comparisson(history_resnet, data_type='Validation Set')
metric_resnet.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 4: Resnet V2 (32) - RGB | 0.455490 | 0.477798 | 0.531250 |
| Model 4: Tuned Resnet V2 (32) - RGB | 0.513667 | 0.506731 | 0.375000 |
Note: You can even go back and build your own architecture on top of the ResNet Transfer layer and see if you can improve the performance.
⏩ Observations and Insights:
⏩ Recommendations:
ap.EfficientNetV2B2(include_top=False, weights="imagenet", input_shape=(48, 48, 3)).summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/efficientnet_v2/efficientnetv2-b2_notop.h5
35839040/35839040 [==============================] - 1s 0us/step
Model: "efficientnetv2-b2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 48, 48, 3)] 0 []
rescaling (Rescaling) (None, 48, 48, 3) 0 ['input_2[0][0]']
normalization (Normalization) (None, 48, 48, 3) 0 ['rescaling[0][0]']
stem_conv (Conv2D) (None, 24, 24, 32) 864 ['normalization[0][0]']
stem_bn (BatchNormalization) (None, 24, 24, 32) 128 ['stem_conv[0][0]']
stem_activation (Activation) (None, 24, 24, 32) 0 ['stem_bn[0][0]']
block1a_project_conv (Conv2D) (None, 24, 24, 16) 4608 ['stem_activation[0][0]']
block1a_project_bn (BatchNorma (None, 24, 24, 16) 64 ['block1a_project_conv[0][0]']
lization)
block1a_project_activation (Ac (None, 24, 24, 16) 0 ['block1a_project_bn[0][0]']
tivation)
block1b_project_conv (Conv2D) (None, 24, 24, 16) 2304 ['block1a_project_activation[0][0
]']
block1b_project_bn (BatchNorma (None, 24, 24, 16) 64 ['block1b_project_conv[0][0]']
lization)
block1b_project_activation (Ac (None, 24, 24, 16) 0 ['block1b_project_bn[0][0]']
tivation)
block1b_drop (Dropout) (None, 24, 24, 16) 0 ['block1b_project_activation[0][0
]']
block1b_add (Add) (None, 24, 24, 16) 0 ['block1b_drop[0][0]',
'block1a_project_activation[0][0
]']
block2a_expand_conv (Conv2D) (None, 12, 12, 64) 9216 ['block1b_add[0][0]']
block2a_expand_bn (BatchNormal (None, 12, 12, 64) 256 ['block2a_expand_conv[0][0]']
ization)
block2a_expand_activation (Act (None, 12, 12, 64) 0 ['block2a_expand_bn[0][0]']
ivation)
block2a_project_conv (Conv2D) (None, 12, 12, 32) 2048 ['block2a_expand_activation[0][0]
']
block2a_project_bn (BatchNorma (None, 12, 12, 32) 128 ['block2a_project_conv[0][0]']
lization)
block2b_expand_conv (Conv2D) (None, 12, 12, 128) 36864 ['block2a_project_bn[0][0]']
block2b_expand_bn (BatchNormal (None, 12, 12, 128) 512 ['block2b_expand_conv[0][0]']
ization)
block2b_expand_activation (Act (None, 12, 12, 128) 0 ['block2b_expand_bn[0][0]']
ivation)
block2b_project_conv (Conv2D) (None, 12, 12, 32) 4096 ['block2b_expand_activation[0][0]
']
block2b_project_bn (BatchNorma (None, 12, 12, 32) 128 ['block2b_project_conv[0][0]']
lization)
block2b_drop (Dropout) (None, 12, 12, 32) 0 ['block2b_project_bn[0][0]']
block2b_add (Add) (None, 12, 12, 32) 0 ['block2b_drop[0][0]',
'block2a_project_bn[0][0]']
block2c_expand_conv (Conv2D) (None, 12, 12, 128) 36864 ['block2b_add[0][0]']
block2c_expand_bn (BatchNormal (None, 12, 12, 128) 512 ['block2c_expand_conv[0][0]']
ization)
block2c_expand_activation (Act (None, 12, 12, 128) 0 ['block2c_expand_bn[0][0]']
ivation)
block2c_project_conv (Conv2D) (None, 12, 12, 32) 4096 ['block2c_expand_activation[0][0]
']
block2c_project_bn (BatchNorma (None, 12, 12, 32) 128 ['block2c_project_conv[0][0]']
lization)
block2c_drop (Dropout) (None, 12, 12, 32) 0 ['block2c_project_bn[0][0]']
block2c_add (Add) (None, 12, 12, 32) 0 ['block2c_drop[0][0]',
'block2b_add[0][0]']
block3a_expand_conv (Conv2D) (None, 6, 6, 128) 36864 ['block2c_add[0][0]']
block3a_expand_bn (BatchNormal (None, 6, 6, 128) 512 ['block3a_expand_conv[0][0]']
ization)
block3a_expand_activation (Act (None, 6, 6, 128) 0 ['block3a_expand_bn[0][0]']
ivation)
block3a_project_conv (Conv2D) (None, 6, 6, 56) 7168 ['block3a_expand_activation[0][0]
']
block3a_project_bn (BatchNorma (None, 6, 6, 56) 224 ['block3a_project_conv[0][0]']
lization)
block3b_expand_conv (Conv2D) (None, 6, 6, 224) 112896 ['block3a_project_bn[0][0]']
block3b_expand_bn (BatchNormal (None, 6, 6, 224) 896 ['block3b_expand_conv[0][0]']
ization)
block3b_expand_activation (Act (None, 6, 6, 224) 0 ['block3b_expand_bn[0][0]']
ivation)
block3b_project_conv (Conv2D) (None, 6, 6, 56) 12544 ['block3b_expand_activation[0][0]
']
block3b_project_bn (BatchNorma (None, 6, 6, 56) 224 ['block3b_project_conv[0][0]']
lization)
block3b_drop (Dropout) (None, 6, 6, 56) 0 ['block3b_project_bn[0][0]']
block3b_add (Add) (None, 6, 6, 56) 0 ['block3b_drop[0][0]',
'block3a_project_bn[0][0]']
block3c_expand_conv (Conv2D) (None, 6, 6, 224) 112896 ['block3b_add[0][0]']
block3c_expand_bn (BatchNormal (None, 6, 6, 224) 896 ['block3c_expand_conv[0][0]']
ization)
block3c_expand_activation (Act (None, 6, 6, 224) 0 ['block3c_expand_bn[0][0]']
ivation)
block3c_project_conv (Conv2D) (None, 6, 6, 56) 12544 ['block3c_expand_activation[0][0]
']
block3c_project_bn (BatchNorma (None, 6, 6, 56) 224 ['block3c_project_conv[0][0]']
lization)
block3c_drop (Dropout) (None, 6, 6, 56) 0 ['block3c_project_bn[0][0]']
block3c_add (Add) (None, 6, 6, 56) 0 ['block3c_drop[0][0]',
'block3b_add[0][0]']
block4a_expand_conv (Conv2D) (None, 6, 6, 224) 12544 ['block3c_add[0][0]']
block4a_expand_bn (BatchNormal (None, 6, 6, 224) 896 ['block4a_expand_conv[0][0]']
ization)
block4a_expand_activation (Act (None, 6, 6, 224) 0 ['block4a_expand_bn[0][0]']
ivation)
block4a_dwconv2 (DepthwiseConv (None, 3, 3, 224) 2016 ['block4a_expand_activation[0][0]
2D) ']
block4a_bn (BatchNormalization (None, 3, 3, 224) 896 ['block4a_dwconv2[0][0]']
)
block4a_activation (Activation (None, 3, 3, 224) 0 ['block4a_bn[0][0]']
)
block4a_se_squeeze (GlobalAver (None, 224) 0 ['block4a_activation[0][0]']
agePooling2D)
block4a_se_reshape (Reshape) (None, 1, 1, 224) 0 ['block4a_se_squeeze[0][0]']
block4a_se_reduce (Conv2D) (None, 1, 1, 14) 3150 ['block4a_se_reshape[0][0]']
block4a_se_expand (Conv2D) (None, 1, 1, 224) 3360 ['block4a_se_reduce[0][0]']
block4a_se_excite (Multiply) (None, 3, 3, 224) 0 ['block4a_activation[0][0]',
'block4a_se_expand[0][0]']
block4a_project_conv (Conv2D) (None, 3, 3, 104) 23296 ['block4a_se_excite[0][0]']
block4a_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4a_project_conv[0][0]']
lization)
block4b_expand_conv (Conv2D) (None, 3, 3, 416) 43264 ['block4a_project_bn[0][0]']
block4b_expand_bn (BatchNormal (None, 3, 3, 416) 1664 ['block4b_expand_conv[0][0]']
ization)
block4b_expand_activation (Act (None, 3, 3, 416) 0 ['block4b_expand_bn[0][0]']
ivation)
block4b_dwconv2 (DepthwiseConv (None, 3, 3, 416) 3744 ['block4b_expand_activation[0][0]
2D) ']
block4b_bn (BatchNormalization (None, 3, 3, 416) 1664 ['block4b_dwconv2[0][0]']
)
block4b_activation (Activation (None, 3, 3, 416) 0 ['block4b_bn[0][0]']
)
block4b_se_squeeze (GlobalAver (None, 416) 0 ['block4b_activation[0][0]']
agePooling2D)
block4b_se_reshape (Reshape) (None, 1, 1, 416) 0 ['block4b_se_squeeze[0][0]']
block4b_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4b_se_reshape[0][0]']
block4b_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4b_se_reduce[0][0]']
block4b_se_excite (Multiply) (None, 3, 3, 416) 0 ['block4b_activation[0][0]',
'block4b_se_expand[0][0]']
block4b_project_conv (Conv2D) (None, 3, 3, 104) 43264 ['block4b_se_excite[0][0]']
block4b_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4b_project_conv[0][0]']
lization)
block4b_drop (Dropout) (None, 3, 3, 104) 0 ['block4b_project_bn[0][0]']
block4b_add (Add) (None, 3, 3, 104) 0 ['block4b_drop[0][0]',
'block4a_project_bn[0][0]']
block4c_expand_conv (Conv2D) (None, 3, 3, 416) 43264 ['block4b_add[0][0]']
block4c_expand_bn (BatchNormal (None, 3, 3, 416) 1664 ['block4c_expand_conv[0][0]']
ization)
block4c_expand_activation (Act (None, 3, 3, 416) 0 ['block4c_expand_bn[0][0]']
ivation)
block4c_dwconv2 (DepthwiseConv (None, 3, 3, 416) 3744 ['block4c_expand_activation[0][0]
2D) ']
block4c_bn (BatchNormalization (None, 3, 3, 416) 1664 ['block4c_dwconv2[0][0]']
)
block4c_activation (Activation (None, 3, 3, 416) 0 ['block4c_bn[0][0]']
)
block4c_se_squeeze (GlobalAver (None, 416) 0 ['block4c_activation[0][0]']
agePooling2D)
block4c_se_reshape (Reshape) (None, 1, 1, 416) 0 ['block4c_se_squeeze[0][0]']
block4c_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4c_se_reshape[0][0]']
block4c_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4c_se_reduce[0][0]']
block4c_se_excite (Multiply) (None, 3, 3, 416) 0 ['block4c_activation[0][0]',
'block4c_se_expand[0][0]']
block4c_project_conv (Conv2D) (None, 3, 3, 104) 43264 ['block4c_se_excite[0][0]']
block4c_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4c_project_conv[0][0]']
lization)
block4c_drop (Dropout) (None, 3, 3, 104) 0 ['block4c_project_bn[0][0]']
block4c_add (Add) (None, 3, 3, 104) 0 ['block4c_drop[0][0]',
'block4b_add[0][0]']
block4d_expand_conv (Conv2D) (None, 3, 3, 416) 43264 ['block4c_add[0][0]']
block4d_expand_bn (BatchNormal (None, 3, 3, 416) 1664 ['block4d_expand_conv[0][0]']
ization)
block4d_expand_activation (Act (None, 3, 3, 416) 0 ['block4d_expand_bn[0][0]']
ivation)
block4d_dwconv2 (DepthwiseConv (None, 3, 3, 416) 3744 ['block4d_expand_activation[0][0]
2D) ']
block4d_bn (BatchNormalization (None, 3, 3, 416) 1664 ['block4d_dwconv2[0][0]']
)
block4d_activation (Activation (None, 3, 3, 416) 0 ['block4d_bn[0][0]']
)
block4d_se_squeeze (GlobalAver (None, 416) 0 ['block4d_activation[0][0]']
agePooling2D)
block4d_se_reshape (Reshape) (None, 1, 1, 416) 0 ['block4d_se_squeeze[0][0]']
block4d_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4d_se_reshape[0][0]']
block4d_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4d_se_reduce[0][0]']
block4d_se_excite (Multiply) (None, 3, 3, 416) 0 ['block4d_activation[0][0]',
'block4d_se_expand[0][0]']
block4d_project_conv (Conv2D) (None, 3, 3, 104) 43264 ['block4d_se_excite[0][0]']
block4d_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4d_project_conv[0][0]']
lization)
block4d_drop (Dropout) (None, 3, 3, 104) 0 ['block4d_project_bn[0][0]']
block4d_add (Add) (None, 3, 3, 104) 0 ['block4d_drop[0][0]',
'block4c_add[0][0]']
block5a_expand_conv (Conv2D) (None, 3, 3, 624) 64896 ['block4d_add[0][0]']
block5a_expand_bn (BatchNormal (None, 3, 3, 624) 2496 ['block5a_expand_conv[0][0]']
ization)
block5a_expand_activation (Act (None, 3, 3, 624) 0 ['block5a_expand_bn[0][0]']
ivation)
block5a_dwconv2 (DepthwiseConv (None, 3, 3, 624) 5616 ['block5a_expand_activation[0][0]
2D) ']
block5a_bn (BatchNormalization (None, 3, 3, 624) 2496 ['block5a_dwconv2[0][0]']
)
block5a_activation (Activation (None, 3, 3, 624) 0 ['block5a_bn[0][0]']
)
block5a_se_squeeze (GlobalAver (None, 624) 0 ['block5a_activation[0][0]']
agePooling2D)
block5a_se_reshape (Reshape) (None, 1, 1, 624) 0 ['block5a_se_squeeze[0][0]']
block5a_se_reduce (Conv2D) (None, 1, 1, 26) 16250 ['block5a_se_reshape[0][0]']
block5a_se_expand (Conv2D) (None, 1, 1, 624) 16848 ['block5a_se_reduce[0][0]']
block5a_se_excite (Multiply) (None, 3, 3, 624) 0 ['block5a_activation[0][0]',
'block5a_se_expand[0][0]']
block5a_project_conv (Conv2D) (None, 3, 3, 120) 74880 ['block5a_se_excite[0][0]']
block5a_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5a_project_conv[0][0]']
lization)
block5b_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5a_project_bn[0][0]']
block5b_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5b_expand_conv[0][0]']
ization)
block5b_expand_activation (Act (None, 3, 3, 720) 0 ['block5b_expand_bn[0][0]']
ivation)
block5b_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5b_expand_activation[0][0]
2D) ']
block5b_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5b_dwconv2[0][0]']
)
block5b_activation (Activation (None, 3, 3, 720) 0 ['block5b_bn[0][0]']
)
block5b_se_squeeze (GlobalAver (None, 720) 0 ['block5b_activation[0][0]']
agePooling2D)
block5b_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5b_se_squeeze[0][0]']
block5b_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5b_se_reshape[0][0]']
block5b_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5b_se_reduce[0][0]']
block5b_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5b_activation[0][0]',
'block5b_se_expand[0][0]']
block5b_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5b_se_excite[0][0]']
block5b_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5b_project_conv[0][0]']
lization)
block5b_drop (Dropout) (None, 3, 3, 120) 0 ['block5b_project_bn[0][0]']
block5b_add (Add) (None, 3, 3, 120) 0 ['block5b_drop[0][0]',
'block5a_project_bn[0][0]']
block5c_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5b_add[0][0]']
block5c_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5c_expand_conv[0][0]']
ization)
block5c_expand_activation (Act (None, 3, 3, 720) 0 ['block5c_expand_bn[0][0]']
ivation)
block5c_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5c_expand_activation[0][0]
2D) ']
block5c_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5c_dwconv2[0][0]']
)
block5c_activation (Activation (None, 3, 3, 720) 0 ['block5c_bn[0][0]']
)
block5c_se_squeeze (GlobalAver (None, 720) 0 ['block5c_activation[0][0]']
agePooling2D)
block5c_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5c_se_squeeze[0][0]']
block5c_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5c_se_reshape[0][0]']
block5c_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5c_se_reduce[0][0]']
block5c_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5c_activation[0][0]',
'block5c_se_expand[0][0]']
block5c_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5c_se_excite[0][0]']
block5c_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5c_project_conv[0][0]']
lization)
block5c_drop (Dropout) (None, 3, 3, 120) 0 ['block5c_project_bn[0][0]']
block5c_add (Add) (None, 3, 3, 120) 0 ['block5c_drop[0][0]',
'block5b_add[0][0]']
block5d_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5c_add[0][0]']
block5d_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5d_expand_conv[0][0]']
ization)
block5d_expand_activation (Act (None, 3, 3, 720) 0 ['block5d_expand_bn[0][0]']
ivation)
block5d_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5d_expand_activation[0][0]
2D) ']
block5d_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5d_dwconv2[0][0]']
)
block5d_activation (Activation (None, 3, 3, 720) 0 ['block5d_bn[0][0]']
)
block5d_se_squeeze (GlobalAver (None, 720) 0 ['block5d_activation[0][0]']
agePooling2D)
block5d_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5d_se_squeeze[0][0]']
block5d_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5d_se_reshape[0][0]']
block5d_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5d_se_reduce[0][0]']
block5d_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5d_activation[0][0]',
'block5d_se_expand[0][0]']
block5d_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5d_se_excite[0][0]']
block5d_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5d_project_conv[0][0]']
lization)
block5d_drop (Dropout) (None, 3, 3, 120) 0 ['block5d_project_bn[0][0]']
block5d_add (Add) (None, 3, 3, 120) 0 ['block5d_drop[0][0]',
'block5c_add[0][0]']
block5e_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5d_add[0][0]']
block5e_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5e_expand_conv[0][0]']
ization)
block5e_expand_activation (Act (None, 3, 3, 720) 0 ['block5e_expand_bn[0][0]']
ivation)
block5e_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5e_expand_activation[0][0]
2D) ']
block5e_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5e_dwconv2[0][0]']
)
block5e_activation (Activation (None, 3, 3, 720) 0 ['block5e_bn[0][0]']
)
block5e_se_squeeze (GlobalAver (None, 720) 0 ['block5e_activation[0][0]']
agePooling2D)
block5e_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5e_se_squeeze[0][0]']
block5e_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5e_se_reshape[0][0]']
block5e_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5e_se_reduce[0][0]']
block5e_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5e_activation[0][0]',
'block5e_se_expand[0][0]']
block5e_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5e_se_excite[0][0]']
block5e_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5e_project_conv[0][0]']
lization)
block5e_drop (Dropout) (None, 3, 3, 120) 0 ['block5e_project_bn[0][0]']
block5e_add (Add) (None, 3, 3, 120) 0 ['block5e_drop[0][0]',
'block5d_add[0][0]']
block5f_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5e_add[0][0]']
block5f_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5f_expand_conv[0][0]']
ization)
block5f_expand_activation (Act (None, 3, 3, 720) 0 ['block5f_expand_bn[0][0]']
ivation)
block5f_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5f_expand_activation[0][0]
2D) ']
block5f_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5f_dwconv2[0][0]']
)
block5f_activation (Activation (None, 3, 3, 720) 0 ['block5f_bn[0][0]']
)
block5f_se_squeeze (GlobalAver (None, 720) 0 ['block5f_activation[0][0]']
agePooling2D)
block5f_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5f_se_squeeze[0][0]']
block5f_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5f_se_reshape[0][0]']
block5f_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5f_se_reduce[0][0]']
block5f_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5f_activation[0][0]',
'block5f_se_expand[0][0]']
block5f_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5f_se_excite[0][0]']
block5f_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5f_project_conv[0][0]']
lization)
block5f_drop (Dropout) (None, 3, 3, 120) 0 ['block5f_project_bn[0][0]']
block5f_add (Add) (None, 3, 3, 120) 0 ['block5f_drop[0][0]',
'block5e_add[0][0]']
block6a_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5f_add[0][0]']
block6a_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block6a_expand_conv[0][0]']
ization)
block6a_expand_activation (Act (None, 3, 3, 720) 0 ['block6a_expand_bn[0][0]']
ivation)
block6a_dwconv2 (DepthwiseConv (None, 2, 2, 720) 6480 ['block6a_expand_activation[0][0]
2D) ']
block6a_bn (BatchNormalization (None, 2, 2, 720) 2880 ['block6a_dwconv2[0][0]']
)
block6a_activation (Activation (None, 2, 2, 720) 0 ['block6a_bn[0][0]']
)
block6a_se_squeeze (GlobalAver (None, 720) 0 ['block6a_activation[0][0]']
agePooling2D)
block6a_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block6a_se_squeeze[0][0]']
block6a_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block6a_se_reshape[0][0]']
block6a_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block6a_se_reduce[0][0]']
block6a_se_excite (Multiply) (None, 2, 2, 720) 0 ['block6a_activation[0][0]',
'block6a_se_expand[0][0]']
block6a_project_conv (Conv2D) (None, 2, 2, 208) 149760 ['block6a_se_excite[0][0]']
block6a_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6a_project_conv[0][0]']
lization)
block6b_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6a_project_bn[0][0]']
block6b_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6b_expand_conv[0][0]']
ization)
block6b_expand_activation (Act (None, 2, 2, 1248) 0 ['block6b_expand_bn[0][0]']
ivation)
block6b_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6b_expand_activation[0][0]
2D) ']
block6b_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6b_dwconv2[0][0]']
)
block6b_activation (Activation (None, 2, 2, 1248) 0 ['block6b_bn[0][0]']
)
block6b_se_squeeze (GlobalAver (None, 1248) 0 ['block6b_activation[0][0]']
agePooling2D)
block6b_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6b_se_squeeze[0][0]']
block6b_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6b_se_reshape[0][0]']
block6b_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6b_se_reduce[0][0]']
block6b_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6b_activation[0][0]',
'block6b_se_expand[0][0]']
block6b_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6b_se_excite[0][0]']
block6b_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6b_project_conv[0][0]']
lization)
block6b_drop (Dropout) (None, 2, 2, 208) 0 ['block6b_project_bn[0][0]']
block6b_add (Add) (None, 2, 2, 208) 0 ['block6b_drop[0][0]',
'block6a_project_bn[0][0]']
block6c_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6b_add[0][0]']
block6c_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6c_expand_conv[0][0]']
ization)
block6c_expand_activation (Act (None, 2, 2, 1248) 0 ['block6c_expand_bn[0][0]']
ivation)
block6c_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6c_expand_activation[0][0]
2D) ']
block6c_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6c_dwconv2[0][0]']
)
block6c_activation (Activation (None, 2, 2, 1248) 0 ['block6c_bn[0][0]']
)
block6c_se_squeeze (GlobalAver (None, 1248) 0 ['block6c_activation[0][0]']
agePooling2D)
block6c_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6c_se_squeeze[0][0]']
block6c_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6c_se_reshape[0][0]']
block6c_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6c_se_reduce[0][0]']
block6c_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6c_activation[0][0]',
'block6c_se_expand[0][0]']
block6c_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6c_se_excite[0][0]']
block6c_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6c_project_conv[0][0]']
lization)
block6c_drop (Dropout) (None, 2, 2, 208) 0 ['block6c_project_bn[0][0]']
block6c_add (Add) (None, 2, 2, 208) 0 ['block6c_drop[0][0]',
'block6b_add[0][0]']
block6d_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6c_add[0][0]']
block6d_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6d_expand_conv[0][0]']
ization)
block6d_expand_activation (Act (None, 2, 2, 1248) 0 ['block6d_expand_bn[0][0]']
ivation)
block6d_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6d_expand_activation[0][0]
2D) ']
block6d_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6d_dwconv2[0][0]']
)
block6d_activation (Activation (None, 2, 2, 1248) 0 ['block6d_bn[0][0]']
)
block6d_se_squeeze (GlobalAver (None, 1248) 0 ['block6d_activation[0][0]']
agePooling2D)
block6d_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6d_se_squeeze[0][0]']
block6d_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6d_se_reshape[0][0]']
block6d_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6d_se_reduce[0][0]']
block6d_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6d_activation[0][0]',
'block6d_se_expand[0][0]']
block6d_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6d_se_excite[0][0]']
block6d_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6d_project_conv[0][0]']
lization)
block6d_drop (Dropout) (None, 2, 2, 208) 0 ['block6d_project_bn[0][0]']
block6d_add (Add) (None, 2, 2, 208) 0 ['block6d_drop[0][0]',
'block6c_add[0][0]']
block6e_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6d_add[0][0]']
block6e_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6e_expand_conv[0][0]']
ization)
block6e_expand_activation (Act (None, 2, 2, 1248) 0 ['block6e_expand_bn[0][0]']
ivation)
block6e_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6e_expand_activation[0][0]
2D) ']
block6e_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6e_dwconv2[0][0]']
)
block6e_activation (Activation (None, 2, 2, 1248) 0 ['block6e_bn[0][0]']
)
block6e_se_squeeze (GlobalAver (None, 1248) 0 ['block6e_activation[0][0]']
agePooling2D)
block6e_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6e_se_squeeze[0][0]']
block6e_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6e_se_reshape[0][0]']
block6e_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6e_se_reduce[0][0]']
block6e_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6e_activation[0][0]',
'block6e_se_expand[0][0]']
block6e_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6e_se_excite[0][0]']
block6e_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6e_project_conv[0][0]']
lization)
block6e_drop (Dropout) (None, 2, 2, 208) 0 ['block6e_project_bn[0][0]']
block6e_add (Add) (None, 2, 2, 208) 0 ['block6e_drop[0][0]',
'block6d_add[0][0]']
block6f_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6e_add[0][0]']
block6f_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6f_expand_conv[0][0]']
ization)
block6f_expand_activation (Act (None, 2, 2, 1248) 0 ['block6f_expand_bn[0][0]']
ivation)
block6f_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6f_expand_activation[0][0]
2D) ']
block6f_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6f_dwconv2[0][0]']
)
block6f_activation (Activation (None, 2, 2, 1248) 0 ['block6f_bn[0][0]']
)
block6f_se_squeeze (GlobalAver (None, 1248) 0 ['block6f_activation[0][0]']
agePooling2D)
block6f_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6f_se_squeeze[0][0]']
block6f_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6f_se_reshape[0][0]']
block6f_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6f_se_reduce[0][0]']
block6f_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6f_activation[0][0]',
'block6f_se_expand[0][0]']
block6f_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6f_se_excite[0][0]']
block6f_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6f_project_conv[0][0]']
lization)
block6f_drop (Dropout) (None, 2, 2, 208) 0 ['block6f_project_bn[0][0]']
block6f_add (Add) (None, 2, 2, 208) 0 ['block6f_drop[0][0]',
'block6e_add[0][0]']
block6g_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6f_add[0][0]']
block6g_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6g_expand_conv[0][0]']
ization)
block6g_expand_activation (Act (None, 2, 2, 1248) 0 ['block6g_expand_bn[0][0]']
ivation)
block6g_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6g_expand_activation[0][0]
2D) ']
block6g_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6g_dwconv2[0][0]']
)
block6g_activation (Activation (None, 2, 2, 1248) 0 ['block6g_bn[0][0]']
)
block6g_se_squeeze (GlobalAver (None, 1248) 0 ['block6g_activation[0][0]']
agePooling2D)
block6g_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6g_se_squeeze[0][0]']
block6g_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6g_se_reshape[0][0]']
block6g_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6g_se_reduce[0][0]']
block6g_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6g_activation[0][0]',
'block6g_se_expand[0][0]']
block6g_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6g_se_excite[0][0]']
block6g_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6g_project_conv[0][0]']
lization)
block6g_drop (Dropout) (None, 2, 2, 208) 0 ['block6g_project_bn[0][0]']
block6g_add (Add) (None, 2, 2, 208) 0 ['block6g_drop[0][0]',
'block6f_add[0][0]']
block6h_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6g_add[0][0]']
block6h_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6h_expand_conv[0][0]']
ization)
block6h_expand_activation (Act (None, 2, 2, 1248) 0 ['block6h_expand_bn[0][0]']
ivation)
block6h_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6h_expand_activation[0][0]
2D) ']
block6h_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6h_dwconv2[0][0]']
)
block6h_activation (Activation (None, 2, 2, 1248) 0 ['block6h_bn[0][0]']
)
block6h_se_squeeze (GlobalAver (None, 1248) 0 ['block6h_activation[0][0]']
agePooling2D)
block6h_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6h_se_squeeze[0][0]']
block6h_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6h_se_reshape[0][0]']
block6h_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6h_se_reduce[0][0]']
block6h_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6h_activation[0][0]',
'block6h_se_expand[0][0]']
block6h_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6h_se_excite[0][0]']
block6h_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6h_project_conv[0][0]']
lization)
block6h_drop (Dropout) (None, 2, 2, 208) 0 ['block6h_project_bn[0][0]']
block6h_add (Add) (None, 2, 2, 208) 0 ['block6h_drop[0][0]',
'block6g_add[0][0]']
block6i_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6h_add[0][0]']
block6i_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6i_expand_conv[0][0]']
ization)
block6i_expand_activation (Act (None, 2, 2, 1248) 0 ['block6i_expand_bn[0][0]']
ivation)
block6i_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6i_expand_activation[0][0]
2D) ']
block6i_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6i_dwconv2[0][0]']
)
block6i_activation (Activation (None, 2, 2, 1248) 0 ['block6i_bn[0][0]']
)
block6i_se_squeeze (GlobalAver (None, 1248) 0 ['block6i_activation[0][0]']
agePooling2D)
block6i_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6i_se_squeeze[0][0]']
block6i_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6i_se_reshape[0][0]']
block6i_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6i_se_reduce[0][0]']
block6i_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6i_activation[0][0]',
'block6i_se_expand[0][0]']
block6i_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6i_se_excite[0][0]']
block6i_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6i_project_conv[0][0]']
lization)
block6i_drop (Dropout) (None, 2, 2, 208) 0 ['block6i_project_bn[0][0]']
block6i_add (Add) (None, 2, 2, 208) 0 ['block6i_drop[0][0]',
'block6h_add[0][0]']
block6j_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6i_add[0][0]']
block6j_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6j_expand_conv[0][0]']
ization)
block6j_expand_activation (Act (None, 2, 2, 1248) 0 ['block6j_expand_bn[0][0]']
ivation)
block6j_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6j_expand_activation[0][0]
2D) ']
block6j_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6j_dwconv2[0][0]']
)
block6j_activation (Activation (None, 2, 2, 1248) 0 ['block6j_bn[0][0]']
)
block6j_se_squeeze (GlobalAver (None, 1248) 0 ['block6j_activation[0][0]']
agePooling2D)
block6j_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6j_se_squeeze[0][0]']
block6j_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6j_se_reshape[0][0]']
block6j_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6j_se_reduce[0][0]']
block6j_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6j_activation[0][0]',
'block6j_se_expand[0][0]']
block6j_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6j_se_excite[0][0]']
block6j_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6j_project_conv[0][0]']
lization)
block6j_drop (Dropout) (None, 2, 2, 208) 0 ['block6j_project_bn[0][0]']
block6j_add (Add) (None, 2, 2, 208) 0 ['block6j_drop[0][0]',
'block6i_add[0][0]']
top_conv (Conv2D) (None, 2, 2, 1408) 292864 ['block6j_add[0][0]']
top_bn (BatchNormalization) (None, 2, 2, 1408) 5632 ['top_conv[0][0]']
top_activation (Activation) (None, 2, 2, 1408) 0 ['top_bn[0][0]']
==================================================================================================
Total params: 8,769,374
Trainable params: 8,687,086
Non-trainable params: 82,288
__________________________________________________________________________________________________
# Model 5: EfficientNet
def model_builder_efficientnet(input_shape, tune=0):
"""Transfer approach with the EfficientNet Model
"""
efficientnet_model = ap.EfficientNetV2B2(include_top=False, weights="imagenet", input_shape=input_shape)
# Making all the layers of the VGG model non-trainable
if tune > 0:
for layer in efficientnet_model.layers[:-tune]:
layer.trainable = False
else:
for layer in efficientnet_model.layers:
layer.trainable = False
model = Sequential([
efficientnet_model,
Flatten(),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dropout(0.3),
Dense(64, activation='relu'),
BatchNormalization(),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_efficientnet, metric_efficientnet, history_efficientnet = model_building_and_evaluating_process(
model_builder_efficientnet, f'Model 5: EfficientNet ({batch_size})', batch_size=batch_size, epochs=20, include_grayscale=False
)
---------------------------------------------
Model 5: EfficientNet (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnetv2-b2 (Function (None, 2, 2, 1408) 8769374
al)
flatten (Flatten) (None, 5632) 0
dense (Dense) (None, 256) 1442048
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 10,253,090
Trainable params: 1,483,588
Non-trainable params: 8,769,502
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 51s 84ms/step - loss: 1.4558 - accuracy: 0.2583 - val_loss: 1.4238 - val_accuracy: 0.1601 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 38s 79ms/step - loss: 1.4049 - accuracy: 0.2586 - val_loss: 1.3468 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 36s 76ms/step - loss: 1.3985 - accuracy: 0.2543 - val_loss: 1.3631 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 36s 77ms/step - loss: 1.3937 - accuracy: 0.2547 - val_loss: 1.3806 - val_accuracy: 0.2443 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 36s 77ms/step - loss: 1.3930 - accuracy: 0.2603 - val_loss: 1.3744 - val_accuracy: 0.2443 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 36s 77ms/step - loss: 1.3860 - accuracy: 0.2605 - val_loss: 1.3871 - val_accuracy: 0.2289 - lr: 2.0000e-04
Epoch 7/20
473/473 [==============================] - 38s 79ms/step - loss: 1.3839 - accuracy: 0.2639 - val_loss: 1.3834 - val_accuracy: 0.3679 - lr: 2.0000e-04
1/1 [==============================] - 0s 74ms/step - loss: 1.4044 - accuracy: 0.3125
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
precision recall f1-score support
happy 0.31 1.00 0.48 10
neutral 0.00 0.00 0.00 5
sad 0.00 0.00 0.00 10
surprise 0.00 0.00 0.00 7
accuracy 0.31 32
macro avg 0.08 0.25 0.12 32
weighted avg 0.10 0.31 0.15 32
# Let's try again tuning some of the last layers in the transfer model
batch_size = 32
model, results, history = model_building_and_evaluating_process(model_builder_efficientnet, f'Model 5: Tuned EfficientNet ({batch_size})',
batch_size=batch_size, epochs=20, tune=7, include_grayscale=False)
model_efficientnet.update(model)
metric_efficientnet = pd.concat([metric_efficientnet, results])
history_efficientnet.update(history)
---------------------------------------------
Model 5: Tuned EfficientNet (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnetv2-b2 (Function (None, 2, 2, 1408) 8769374
al)
flatten (Flatten) (None, 5632) 0
dense (Dense) (None, 256) 1442048
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 10,253,090
Trainable params: 2,039,268
Non-trainable params: 8,213,822
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 54s 90ms/step - loss: 1.4554 - accuracy: 0.2542 - val_loss: 1.3559 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 38s 80ms/step - loss: 1.3994 - accuracy: 0.2585 - val_loss: 1.3627 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 38s 81ms/step - loss: 1.3935 - accuracy: 0.2574 - val_loss: 1.3700 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 39s 83ms/step - loss: 1.3933 - accuracy: 0.2640 - val_loss: 1.3663 - val_accuracy: 0.3671 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 38s 81ms/step - loss: 1.3865 - accuracy: 0.2595 - val_loss: 1.3703 - val_accuracy: 0.2289 - lr: 2.0000e-04
Epoch 6/20
473/473 [==============================] - 39s 82ms/step - loss: 1.3851 - accuracy: 0.2637 - val_loss: 1.3658 - val_accuracy: 0.3667 - lr: 2.0000e-04
1/1 [==============================] - 0s 80ms/step - loss: 1.3944 - accuracy: 0.3125
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
precision recall f1-score support
happy 0.31 1.00 0.48 10
neutral 0.00 0.00 0.00 5
sad 0.00 0.00 0.00 10
surprise 0.00 0.00 0.00 7
accuracy 0.31 32
macro avg 0.08 0.25 0.12 32
weighted avg 0.10 0.31 0.15 32
model_name = 'Model 5: EfficientNet'
save_object(history_efficientnet, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_efficientnet, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_efficientnet, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_efficientnet, data_type='Training Set')
loss_and_accuracy_comparisson(history_efficientnet, data_type='Validation Set')
metric_efficientnet.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 5: EfficientNet (32) - RGB | 0.263949 | 0.367892 | 0.312500 |
| Model 5: Tuned EfficientNet (32) - RGB | 0.264015 | 0.367089 | 0.312500 |
Note: You can even go back and build your own architecture on top of the VGG16 Transfer layer and see if you can improve the performance.
⏩ Observations and Insights:
⏩ Recommendations:
Think About It:
# Getting back the history and metrics result during trainning
n = 'Model 3: VGG16'
with open(f'{MODEL_DIR}/{n} - history.pkl', 'rb') as f: history_vgg16 = pickle.load(f)
with open(f'{MODEL_DIR}/{n} - metrics.pkl', 'rb') as f: metric_vgg16 = pickle.load(f)
n = 'Model 4: Resnet V2'
with open(f'{MODEL_DIR}/{n} - history.pkl', 'rb') as f: history_resnet = pickle.load(f)
with open(f'{MODEL_DIR}/{n} - metrics.pkl', 'rb') as f: metric_resnet = pickle.load(f)
n = 'Model 5: EfficientNet'
with open(f'{MODEL_DIR}/{n} - history.pkl', 'rb') as f: history_efficientnet = pickle.load(f)
with open(f'{MODEL_DIR}/{n} - metrics.pkl', 'rb') as f: metric_efficientnet = pickle.load(f)
history_transfer_models = {
'Model 3: Tuned VGG16 (32) - RGB': history_vgg16['Model 3: Tuned VGG16 (32) - RGB'],
'Model 4: Tuned Resnet V2 (32) - RGB': history_resnet['Model 4: Tuned Resnet V2 (32) - RGB'],
'Model 5: Tuned EfficientNet (32) - RGB': history_efficientnet['Model 5: Tuned EfficientNet (32) - RGB']
}
loss_and_accuracy_comparisson(history_transfer_models, data_type='Training Set')
loss_and_accuracy_comparisson(history_transfer_models, data_type='Validation Set')
metric_transfer_models = pd.concat([
metric_vgg16.loc['Model 3: Tuned VGG16 (32) - RGB':],
metric_resnet.loc['Model 4: Tuned Resnet V2 (32) - RGB':],
metric_efficientnet.loc['Model 5: Tuned EfficientNet (32) - RGB':]
])
metric_transfer_models
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 3: Tuned VGG16 (32) - RGB | 0.784367 | 0.712879 | 0.8125 |
| Model 4: Tuned Resnet V2 (32) - RGB | 0.513667 | 0.506731 | 0.3750 |
| Model 5: Tuned EfficientNet (32) - RGB | 0.264015 | 0.367089 | 0.3125 |
_ = metric_transfer_models.plot(kind='bar', figsize=(10,3))
plt.xticks(rotation=0, fontsize=8)
plt.yticks(fontsize=8)
plt.title('Best Performed Trained Transfer Models', fontsize=10)
plt.legend(fontsize=8)
plt.show()
pd.concat([metric_vgg16, metric_resnet, metric_efficientnet]).plot(kind='bar', figsize=(15,3))
plt.xticks(rotation=0, fontsize=8)
plt.yticks(fontsize=8)
plt.title('Trained Transfer Models', fontsize=10)
plt.legend(fontsize=8)
plt.show()
⏩ Observations and Insights:
Now that we have tried multiple pre-trained models, let's build a complex CNN architecture and see if we can get better performance.
In this section, we will build a more complex Convolutional Neural Network Model that has close to as many parameters as we had in our Transfer Learning Models. However, we will have only 1 input channel for our input images.
In this section, we are creating data loaders which we will use as inputs to the more Complicated Convolutional Neural Network. We will go ahead with color_mode = 'grayscale'.
train_generator_gray, _, _ = create_data_loaders(IMG_DIR, 'grayscale')
# Taking a look at some examples of our augmented training data with color_mode='rgb'.
images, labels = next(train_generator_gray)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image, cmap='gray')
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=grayscale: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
# Model 6: Complex Neural Network
def model_builder_complex_cnn(input_shape):
"""Creating a Complex Neural Network.
"""
model = Sequential([
Conv2D(filters=64, kernel_size=2, activation='relu', padding='same', input_shape=input_shape),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=128, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=512, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=512, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=128, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Flatten(),
Dense(256, activation='relu'),
BatchNormalization(),
Dropout(0.4),
Dense(512, activation='relu'),
BatchNormalization(),
Dropout(0.4),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.01) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_complex_cnn, metric_complex_cnn, history_complex_cnn = model_building_and_evaluating_process(
model_builder_complex_cnn, f'Model 6: Complex CNN ({batch_size})', batch_size=batch_size, epochs=20, include_rgb=False
)
---------------------------------------------
Model 6: Complex CNN (32) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 44s 83ms/step - loss: 1.7116 - accuracy: 0.2803 - val_loss: 1.5907 - val_accuracy: 0.2536 - lr: 0.0100
Epoch 2/20
473/473 [==============================] - 39s 82ms/step - loss: 1.3762 - accuracy: 0.3332 - val_loss: 1.2800 - val_accuracy: 0.3940 - lr: 0.0100
Epoch 3/20
473/473 [==============================] - 39s 82ms/step - loss: 1.2049 - accuracy: 0.4303 - val_loss: 1.2257 - val_accuracy: 0.4577 - lr: 0.0100
Epoch 4/20
473/473 [==============================] - 39s 82ms/step - loss: 1.0813 - accuracy: 0.5174 - val_loss: 1.0792 - val_accuracy: 0.5248 - lr: 0.0100
Epoch 5/20
473/473 [==============================] - 39s 82ms/step - loss: 1.0102 - accuracy: 0.5609 - val_loss: 0.9836 - val_accuracy: 0.5817 - lr: 0.0100
Epoch 6/20
473/473 [==============================] - 38s 81ms/step - loss: 0.9574 - accuracy: 0.5900 - val_loss: 1.1192 - val_accuracy: 0.5357 - lr: 0.0100
Epoch 7/20
473/473 [==============================] - 38s 80ms/step - loss: 0.9274 - accuracy: 0.6116 - val_loss: 1.1068 - val_accuracy: 0.5043 - lr: 0.0100
Epoch 8/20
473/473 [==============================] - 39s 83ms/step - loss: 0.9071 - accuracy: 0.6203 - val_loss: 0.8265 - val_accuracy: 0.6536 - lr: 0.0100
Epoch 9/20
473/473 [==============================] - 38s 81ms/step - loss: 0.8800 - accuracy: 0.6366 - val_loss: 0.8600 - val_accuracy: 0.6564 - lr: 0.0100
Epoch 10/20
473/473 [==============================] - 38s 81ms/step - loss: 0.8668 - accuracy: 0.6406 - val_loss: 0.8620 - val_accuracy: 0.6323 - lr: 0.0100
Epoch 11/20
473/473 [==============================] - 38s 80ms/step - loss: 0.8432 - accuracy: 0.6469 - val_loss: 3.5923 - val_accuracy: 0.5337 - lr: 0.0100
Epoch 12/20
473/473 [==============================] - 39s 82ms/step - loss: 0.7678 - accuracy: 0.6855 - val_loss: 0.6736 - val_accuracy: 0.7281 - lr: 0.0020
Epoch 13/20
473/473 [==============================] - 38s 81ms/step - loss: 0.7385 - accuracy: 0.7013 - val_loss: 0.6852 - val_accuracy: 0.7231 - lr: 0.0020
Epoch 14/20
473/473 [==============================] - 38s 81ms/step - loss: 0.7189 - accuracy: 0.7120 - val_loss: 0.6652 - val_accuracy: 0.7275 - lr: 0.0020
Epoch 15/20
473/473 [==============================] - 38s 81ms/step - loss: 0.7087 - accuracy: 0.7169 - val_loss: 0.6839 - val_accuracy: 0.7241 - lr: 0.0020
Epoch 16/20
473/473 [==============================] - 39s 82ms/step - loss: 0.6888 - accuracy: 0.7221 - val_loss: 0.6374 - val_accuracy: 0.7434 - lr: 0.0020
Epoch 17/20
473/473 [==============================] - 39s 83ms/step - loss: 0.6760 - accuracy: 0.7274 - val_loss: 0.6516 - val_accuracy: 0.7326 - lr: 0.0020
Epoch 18/20
473/473 [==============================] - 40s 84ms/step - loss: 0.6656 - accuracy: 0.7339 - val_loss: 0.6614 - val_accuracy: 0.7296 - lr: 0.0020
Epoch 19/20
473/473 [==============================] - 40s 84ms/step - loss: 0.6519 - accuracy: 0.7387 - val_loss: 0.6358 - val_accuracy: 0.7472 - lr: 0.0020
Epoch 20/20
473/473 [==============================] - 38s 81ms/step - loss: 0.6468 - accuracy: 0.7395 - val_loss: 0.7484 - val_accuracy: 0.6908 - lr: 0.0020
1/1 [==============================] - 0s 38ms/step - loss: 0.7757 - accuracy: 0.7500
1/1 [==============================] - 0s 199ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 2 3 0 2 3 2 1 0 2 1 3 2 0 3 0 3 2 0 2 2 2 2 2 0 2 1 0 1 2 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.50 0.40 0.44 5
sad 0.64 0.90 0.75 10
surprise 1.00 0.71 0.83 7
accuracy 0.75 32
macro avg 0.76 0.70 0.72 32
weighted avg 0.78 0.75 0.75 32
batch_size = 64
model, results, history = model_building_and_evaluating_process(model_builder_complex_cnn, f'Model 6: Complex CNN ({batch_size})',
batch_size=batch_size, epochs=20, include_rgb=False)
model_complex_cnn.update(model)
metric_complex_cnn = pd.concat([metric_complex_cnn, results])
history_complex_cnn.update(history)
---------------------------------------------
Model 6: Complex CNN (64) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 42s 163ms/step - loss: 1.7046 - accuracy: 0.2904 - val_loss: 1.4634 - val_accuracy: 0.2443 - lr: 0.0100
Epoch 2/20
237/237 [==============================] - 38s 159ms/step - loss: 1.4348 - accuracy: 0.3182 - val_loss: 1.3236 - val_accuracy: 0.3430 - lr: 0.0100
Epoch 3/20
237/237 [==============================] - 38s 159ms/step - loss: 1.2923 - accuracy: 0.3857 - val_loss: 1.2863 - val_accuracy: 0.3436 - lr: 0.0100
Epoch 4/20
237/237 [==============================] - 38s 158ms/step - loss: 1.1288 - accuracy: 0.4879 - val_loss: 1.1257 - val_accuracy: 0.5224 - lr: 0.0100
Epoch 5/20
237/237 [==============================] - 37s 156ms/step - loss: 1.0201 - accuracy: 0.5523 - val_loss: 1.1093 - val_accuracy: 0.4999 - lr: 0.0100
Epoch 6/20
237/237 [==============================] - 37s 157ms/step - loss: 0.9889 - accuracy: 0.5671 - val_loss: 1.4645 - val_accuracy: 0.4836 - lr: 0.0100
Epoch 7/20
237/237 [==============================] - 38s 159ms/step - loss: 0.9374 - accuracy: 0.6017 - val_loss: 0.9721 - val_accuracy: 0.5893 - lr: 0.0100
Epoch 8/20
237/237 [==============================] - 37s 156ms/step - loss: 0.9039 - accuracy: 0.6202 - val_loss: 1.1374 - val_accuracy: 0.4933 - lr: 0.0100
Epoch 9/20
237/237 [==============================] - 38s 160ms/step - loss: 0.9106 - accuracy: 0.6167 - val_loss: 0.7781 - val_accuracy: 0.6827 - lr: 0.0100
Epoch 10/20
237/237 [==============================] - 37s 156ms/step - loss: 0.8654 - accuracy: 0.6390 - val_loss: 0.9844 - val_accuracy: 0.5897 - lr: 0.0100
Epoch 11/20
237/237 [==============================] - 37s 156ms/step - loss: 0.8498 - accuracy: 0.6473 - val_loss: 0.9861 - val_accuracy: 0.5586 - lr: 0.0100
Epoch 12/20
237/237 [==============================] - 37s 157ms/step - loss: 0.8299 - accuracy: 0.6605 - val_loss: 0.8312 - val_accuracy: 0.6508 - lr: 0.0100
Epoch 13/20
237/237 [==============================] - 38s 159ms/step - loss: 0.7649 - accuracy: 0.6856 - val_loss: 0.7060 - val_accuracy: 0.7201 - lr: 0.0020
Epoch 14/20
237/237 [==============================] - 38s 159ms/step - loss: 0.7394 - accuracy: 0.6950 - val_loss: 0.6770 - val_accuracy: 0.7306 - lr: 0.0020
Epoch 15/20
237/237 [==============================] - 37s 155ms/step - loss: 0.7169 - accuracy: 0.7086 - val_loss: 0.6951 - val_accuracy: 0.7199 - lr: 0.0020
Epoch 16/20
237/237 [==============================] - 37s 156ms/step - loss: 0.7073 - accuracy: 0.7137 - val_loss: 0.7032 - val_accuracy: 0.7195 - lr: 0.0020
Epoch 17/20
237/237 [==============================] - 37s 156ms/step - loss: 0.6990 - accuracy: 0.7155 - val_loss: 0.7149 - val_accuracy: 0.7145 - lr: 0.0020
Epoch 18/20
237/237 [==============================] - 37s 158ms/step - loss: 0.6783 - accuracy: 0.7285 - val_loss: 0.6455 - val_accuracy: 0.7408 - lr: 4.0000e-04
Epoch 19/20
237/237 [==============================] - 37s 155ms/step - loss: 0.6710 - accuracy: 0.7309 - val_loss: 0.6517 - val_accuracy: 0.7392 - lr: 4.0000e-04
Epoch 20/20
237/237 [==============================] - 38s 158ms/step - loss: 0.6648 - accuracy: 0.7282 - val_loss: 0.6724 - val_accuracy: 0.7410 - lr: 4.0000e-04
1/1 [==============================] - 0s 38ms/step - loss: 0.7054 - accuracy: 0.6875
1/1 [==============================] - 0s 196ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 0 3 0 3 2 0 2 2 2 1 1 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.90 0.90 0.90 10
neutral 0.38 0.60 0.46 5
sad 0.56 0.50 0.53 10
surprise 1.00 0.71 0.83 7
accuracy 0.69 32
macro avg 0.71 0.68 0.68 32
weighted avg 0.73 0.69 0.70 32
batch_size = 128
model, results, history = model_building_and_evaluating_process(model_builder_complex_cnn, f'Model 6: Complex CNN ({batch_size})',
batch_size=batch_size, epochs=20, include_rgb=False)
model_complex_cnn.update(model)
metric_complex_cnn = pd.concat([metric_complex_cnn, results])
history_complex_cnn.update(history)
---------------------------------------------
Model 6: Complex CNN (128) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 41s 316ms/step - loss: 1.7298 - accuracy: 0.2822 - val_loss: 1.5841 - val_accuracy: 0.3617 - lr: 0.0100
Epoch 2/20
119/119 [==============================] - 37s 309ms/step - loss: 1.5195 - accuracy: 0.2928 - val_loss: 2.0473 - val_accuracy: 0.2849 - lr: 0.0100
Epoch 3/20
119/119 [==============================] - 37s 311ms/step - loss: 1.4196 - accuracy: 0.3137 - val_loss: 1.3419 - val_accuracy: 0.3749 - lr: 0.0100
Epoch 4/20
119/119 [==============================] - 37s 308ms/step - loss: 1.3748 - accuracy: 0.3305 - val_loss: 1.4027 - val_accuracy: 0.3669 - lr: 0.0100
Epoch 5/20
119/119 [==============================] - 38s 316ms/step - loss: 1.3493 - accuracy: 0.3386 - val_loss: 1.3150 - val_accuracy: 0.3783 - lr: 0.0100
Epoch 6/20
119/119 [==============================] - 38s 315ms/step - loss: 1.3065 - accuracy: 0.3704 - val_loss: 1.2485 - val_accuracy: 0.4137 - lr: 0.0100
Epoch 7/20
119/119 [==============================] - 38s 315ms/step - loss: 1.2580 - accuracy: 0.4076 - val_loss: 1.2150 - val_accuracy: 0.4505 - lr: 0.0100
Epoch 8/20
119/119 [==============================] - 38s 315ms/step - loss: 1.1512 - accuracy: 0.4814 - val_loss: 1.1007 - val_accuracy: 0.5101 - lr: 0.0100
Epoch 9/20
119/119 [==============================] - 38s 315ms/step - loss: 1.0675 - accuracy: 0.5256 - val_loss: 1.0095 - val_accuracy: 0.5660 - lr: 0.0100
Epoch 10/20
119/119 [==============================] - 38s 315ms/step - loss: 0.9822 - accuracy: 0.5731 - val_loss: 1.0215 - val_accuracy: 0.5769 - lr: 0.0100
Epoch 11/20
119/119 [==============================] - 37s 311ms/step - loss: 0.9774 - accuracy: 0.5779 - val_loss: 1.3275 - val_accuracy: 0.4762 - lr: 0.0100
Epoch 12/20
119/119 [==============================] - 37s 310ms/step - loss: 0.9699 - accuracy: 0.5875 - val_loss: 1.1155 - val_accuracy: 0.5634 - lr: 0.0100
Epoch 13/20
119/119 [==============================] - 38s 315ms/step - loss: 0.8904 - accuracy: 0.6245 - val_loss: 0.8109 - val_accuracy: 0.6649 - lr: 0.0020
Epoch 14/20
119/119 [==============================] - 37s 314ms/step - loss: 0.8687 - accuracy: 0.6342 - val_loss: 0.7978 - val_accuracy: 0.6701 - lr: 0.0020
Epoch 15/20
119/119 [==============================] - 38s 316ms/step - loss: 0.8469 - accuracy: 0.6472 - val_loss: 0.7810 - val_accuracy: 0.6751 - lr: 0.0020
Epoch 16/20
119/119 [==============================] - 37s 313ms/step - loss: 0.8385 - accuracy: 0.6521 - val_loss: 1.3783 - val_accuracy: 0.3994 - lr: 0.0020
Epoch 17/20
119/119 [==============================] - 38s 316ms/step - loss: 0.8340 - accuracy: 0.6547 - val_loss: 0.7525 - val_accuracy: 0.6906 - lr: 0.0020
Epoch 18/20
119/119 [==============================] - 37s 310ms/step - loss: 0.8155 - accuracy: 0.6619 - val_loss: 0.8422 - val_accuracy: 0.6701 - lr: 0.0020
Epoch 19/20
119/119 [==============================] - 37s 310ms/step - loss: 0.8082 - accuracy: 0.6675 - val_loss: 0.7609 - val_accuracy: 0.6866 - lr: 0.0020
Epoch 20/20
119/119 [==============================] - 37s 313ms/step - loss: 0.7919 - accuracy: 0.6761 - val_loss: 0.7667 - val_accuracy: 0.6878 - lr: 0.0020
1/1 [==============================] - 0s 37ms/step - loss: 0.8769 - accuracy: 0.5938
1/1 [==============================] - 0s 195ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 2 3 2 0 3 0 3 2 2 2 2 2 1 1 0 2 2 0 0 1 0 2]
precision recall f1-score support
happy 0.78 0.70 0.74 10
neutral 0.33 0.40 0.36 5
sad 0.42 0.50 0.45 10
surprise 1.00 0.71 0.83 7
accuracy 0.59 32
macro avg 0.63 0.58 0.60 32
weighted avg 0.64 0.59 0.61 32
model_name = 'Model 6: Complex CNN'
save_object(history_complex_cnn, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_complex_cnn, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_complex_cnn, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_complex_cnn, data_type='Training Set')
loss_and_accuracy_comparisson(history_complex_cnn, data_type='Validation Set')
metric_complex_cnn.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 6: Complex CNN (32) - GRAY | 0.738699 | 0.747237 | 0.750000 |
| Model 6: Complex CNN (64) - GRAY | 0.728175 | 0.741009 | 0.687500 |
| Model 6: Complex CNN (128) - GRAY | 0.654709 | 0.690577 | 0.593750 |
⏩ Observations and Insights:
⏩ Recommendations:
# Getting back the history and metrics result during trainning
with open(f'{MODEL_DIR}/Model 1: Base Neural Network - metrics.pkl', 'rb') as f: metric_base_cnn = pickle.load(f)
with open(f'{MODEL_DIR}/Model 2: Enhanced Neural Network - metrics.pkl', 'rb') as f: metric_enhanced_cnn = pickle.load(f)
with open(f'{MODEL_DIR}/Model 3: VGG16 - metrics.pkl', 'rb') as f: metric_vgg16 = pickle.load(f)
with open(f'{MODEL_DIR}/Model 4: Resnet V2 - metrics.pkl', 'rb') as f: metric_resnet = pickle.load(f)
with open(f'{MODEL_DIR}/Model 5: EfficientNet - metrics.pkl', 'rb') as f: metric_efficientnet = pickle.load(f)
with open(f'{MODEL_DIR}/Model 6: Complex CNN - metrics.pkl', 'rb') as f: metric_complex_cnn = pickle.load(f)
best_performed_models = pd.concat([
metric_base_cnn.loc['Model 1: Base Neural Network (32) - RGB':'Model 1: Base Neural Network (32) - RGB'],
metric_enhanced_cnn['Model 2: Enhanced Neural Network (64) - RGB':'Model 2: Enhanced Neural Network (64) - RGB'],
metric_vgg16.loc['Model 3: Tuned VGG16 (32) - RGB':'Model 3: Tuned VGG16 (32) - RGB'],
metric_resnet.loc['Model 4: Tuned Resnet V2 (32) - RGB':'Model 4: Tuned Resnet V2 (32) - RGB'],
metric_efficientnet.loc['Model 5: EfficientNet (32) - RGB':'Model 5: EfficientNet (32) - RGB'],
metric_complex_cnn.loc['Model 6: Complex CNN (32) - GRAY':'Model 6: Complex CNN (32) - GRAY']
])
best_performed_models
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 1: Base Neural Network (32) - RGB | 0.705871 | 0.719108 | 0.78125 |
| Model 2: Enhanced Neural Network (64) - RGB | 0.826593 | 0.736789 | 0.68750 |
| Model 3: Tuned VGG16 (32) - RGB | 0.784367 | 0.712879 | 0.81250 |
| Model 4: Tuned Resnet V2 (32) - RGB | 0.513667 | 0.506731 | 0.37500 |
| Model 5: EfficientNet (32) - RGB | 0.263949 | 0.367892 | 0.31250 |
| Model 6: Complex CNN (32) - GRAY | 0.738699 | 0.747237 | 0.75000 |
_ = best_performed_models.plot(kind='bar', figsize=(20,3))
plt.xticks(rotation=0, fontsize=8)
plt.axhline(y=0.74, lw=1, ls='--', c='red')
plt.yticks(fontsize=6)
plt.title('Best Performed Models', fontsize=10)
plt.legend(fontsize=8)
plt.show()
# Getting back a model
# Preparing the environment
init_environment()
_, test_generator, _ = create_data_loaders(IMG_DIR, 'grayscale')
X_test, y_test = test_generator.next()
# Loading the model
model = model_builder_complex_cnn(input_shape=(48, 48, 1))
model.load_weights(f'{MODEL_DIR}/Model 6: Complex CNN (32) - GRAY.h5')
# Re-evaluate the model
loss, acc = model.evaluate(X_test, y_test, verbose=2)
model.evaluate(X_test, y_test, verbose=1)
print("Restored model, accuracy: {:5.2f}%".format(100 * acc))
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
1/1 - 0s - loss: 0.7360 - accuracy: 0.7500 - 495ms/epoch - 495ms/step
1/1 [==============================] - 0s 40ms/step - loss: 0.7360 - accuracy: 0.7500
Restored model, accuracy: 75.00%
# Getting the history back as this a required parameter in our defined function.
with open(f'{MODEL_DIR}/Model 6: Complex CNN - history.pkl', 'rb') as f: history = pickle.load(f)
# Preparing the variables
y_pred = np.argmax(model.predict(X_test), axis=1)
y_test = np.argmax(y_test, axis=1)
print('Actual :', y_test)
print('Prediction:', y_pred)
print('\n\n')
# Plotting the classification report and confussion matrix
results = metrics_score(y_test, y_pred, 'Model 6: Complex CNN (32) - GRAY', history['Model 6: Complex CNN (32) - GRAY'])
1/1 [==============================] - 0s 216ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 1 3 2 0 3 0 3 2 0 2 1 3 1 2 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.44 0.80 0.57 5
sad 0.75 0.60 0.67 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.77 0.76 0.75 32
weighted avg 0.80 0.75 0.76 32
⏩ Observations and Insights:
⏩ Observations and Insights:
A couple of things can be done to try to improve this model:
⏩ Conclusions and Insights:
What are the most meaningful insights from the data relevant to the problem?
⏩ Even do, the dataset is balanced, features in some of the classes are difficult to identify, even for the human eyes. A review of the training dataset is necessary to confirm data is good enough. As it was mentioned before some images seem to belong to another class that the one they are currently on.
⏩ With the exception of the "Model 4: Tuned Resnet V2" and "Model 5: Tuned EfficientNet", all models achieve more than 70% of accuracy. However, the time to tune and explore each model was huge.
⏩ "Model 4: Tuned Resnet V2" and "Model 5: Tuned EfficientNet" need to be recompiled with the "tune" parameter increased to enable a longer backpropagation, exploring if this improves the model.
⏩ There are 3 models in the final solution that can be selected:
- "Model 2: Enhanced Neural Network (64) - RGB" that achieved 73.7% of accuracy.
- "Model 3: Tuned VGG16 (32) - RGB" that achieved 71.3% of accuracy, had a consistent evolution through epochs during the training, and took advantage of the VGG16 models.
- "Model 6: Complex CNN (32) - GRAY" that achieved 74.7% of accuracy, and had a good performance on the testing set.
⏩ With the current trained models and results, I recommend "Model 6: Complex CNN (32) - GRAY" as it achieved the best validation performance.
⏩ In this classification problem, the goal was to find a model that could classify images in 4 different expressions: Happy, Surprise, Sad, and Neutral. For that, I have experimented with 21 different models and 3 approaches to try to find the best model that has an acceptable behavior in this classification task. Accordingly to the business, this task is quite important and is part of the roadmap they have in place for the present period. The main specification was to find a model that could classify the images better than a random guess, meaning that at least a 70% of accuracy would be acceptable for a proposed model.
⏩ There were 21 models that were trained and reviewed:
- 6 models related to a basic CNN model of 13 layers. These models were trained considering the batch sizes 32, 64, and 128, and the images set in RGB or Grayscale. The best result was obtained with the batch size 32 and the RGB set. These models included, in the convolutional layer block, the following layers: Conv2D, max_pooling2d (for downsampling the feature map), and dropout (for dealing with possible overfitting).
- 6 models related to an enhanced CNN model of 20 layers. These models were trained considering the batch sizes 32, 64, and 128, and the images set in RGB or Grayscale. The best result was obtained with the batch size 64 and the RGB set. These models were a little bit more complex than models 1, 2, and 3. They included "leaky_re_lu" (to ensure positive values always, an enhancement of the normal ReLu activation function), "batch_normalization" (to speed up training and use higher learning rates, making learning easier), and "max_pooling2d" (to downsample the feature map).
- 2 models using the transfer model approach with VGG16. This model considered the 18 layers of the convolutional VGG16 block and 7 layers added as part of the classification component. The first try saved/applied the already achieved weights of the VGG16 convolutional block, and in the second try, a tuned process took place, retraining the last 4 layers of the convolutional block (VGG16) with the training dataset belonging to the current classification problem. The best result was obtained with this tunned VGG16 transfer model.
- 2 models using the transfer model approach with ResNet V2. This model considered the 546 layers of the convolutional ResNet V2 block and 7 layers added as part of the classification component. The first try saved/applied the already achieved weights of the ResNet V2 convolutional block, and in the second try, a tunned process took place, retraining the last 5 layers of that convolutional block (ResNet V2) with the training dataset belonging to the current classification problem. The best result was obtained with this tuned ResNet V2 transfer model.
- 2 models using the transfer model approach with EfficientNet. This model includes the 561 layers of the convolutional EfficientNet block and 7 layers added as part of the classification component. The first try saved/applied the already achieved weights of the EfficientNet convolutional block, and in the second try, a tuned process took place, retraining the last 7 layers of that convolutional block (EfficientNet) with the training dataset belonging to the current classification problem. The best result was obtained with the no-tuned EfficientNet transfer model.
- 3 models related to a Complex CNN model of 33 layers and a batch size of 32, 64, and 128, considering only the images set in grayscale. These models were more complex, a kind of robust CNN model, They included "Conv2D", "leaky_re_lu" (to ensure positive values always, an enhancement of the normal ReLu activation function), "batch_normalization" (to speed up training and enable higher learning rates), "max_pooling2d" (to downsample the feature map), and dropout (to deal with possible overfitting). The best result was obtained with batch size 32.
⏩ The 3 followed approaches were:
- A basic CNN model (Models from 1 to 6) that explored the use of the convolutional layers and how to deal with overfitting during the training.
- A transfer model technique (Models from 7 to 12), to use the advantages of already training models and use those achieved weights to include them as part of the current classification problem.
- A complex CNN model (Models from 13 to 15) that applies the learned lessons on the previous approach to finding a path that can achieve the expected result in the classification problem.
Findings
- Complex models lead to having a better understanding of the data in classification problems. Overfitting is the highest risk in complex models. -Transfer models are an optimal approach to take advantage of models trained in millions of images, saving computation resources, however, overfitting is also present, and techniques to deal with this problem need to be considered/applied. Also, it is necessary to enable backpropagation in the final layers of the convolutional block to enable the model to learn the particularities of the problem and get the best of it.
- The quality of the dataset is a key player in the modeling of a solution, bad poor data set leads to bad poor models.
Final Proposed Model
⏩ The model that achieved the greatest accuracy was the "Complex model with batch size 32 and focused on a grayscale images dataset". In only 20 epochs this model achieved approximately 75% of accuracy, looking promising for the future.
The selected and recommended model has 33 layers, the first 25 layers belong to the Feature extraction, and the last 8 layers to the Classification process. The feature extraction section is conformed of 5 convolutional blocks, each of them considered Conv2D, BatchNormalization, LeakyReLU, MaxPooling2D, and Dropout. The classification section related to the fully connected layers considered Flatten, Dense, BatchNormalization, and Dropout. Notice that Overfitting was considered and addressed in this complex robust CNN model.
⏩ The "Complex model with batch size 32 and focused on a grayscale images dataset" is the selected and recommended model because:
- Achieved the best result in all the 21 training models.
- In only 20 epochs reached 75% of accuracy.
- The learning curve shows that learning is still possible, so having more epochs to train will be beneficial.
⏩ The recommended model "Complex model with batch size 32 and focused on a grayscale images dataset" can be applied to the business to start classifying the images to address each image according to its class, this will improve the business operation by reducing time and cost. The company will not require hiring people for classification activities, allowing them to focus on their main objectives that take place after addressing each image in the right direction.
Benefits¶
⏩ Making a comparison between the cost and time invested in the classification of 30,000 images, we have:
⏩ For the Manual Process:
- Invested time: 21 working days.
20 sec. per image, for 30,000 images, one person will need 21 working days to complete the task.- Operational cost: 1,218.00 USD per month. No more tasks can be assigned.
Considering the minimum hourly salary of 7.25 USD, the cost of having one person working on the classification of 30,000 images will cost 1,218 USD.⏩ For the Deep Learning Supported Process
- Invested time: 7 minutes.
Approx 0.5 sec per batch of 32 images using Google Colab Pro Service. To classify 30,000 images, the model will need approx 7 minutes.- Operational cost: 9.99 USD per month. More tasks can be assigned.
The Google Colab Pro Service costs 9.99 USD per month. This provides 100 compute units per month, Faster GPUs, and access to higher memory machines.⏩ If the business implements this recommended Model, it will save time and money on this classification task, allowing them to focus resources on more strategic activities.
⏩ Next steps and recommendations
- All previous models achieved better results with the RGB datasets, exploring the recommended model with the RGB dataset is advisable to verify if the model can improve.
- The learning curve shows that the recommended model still learns at the end of the 20 epochs, it is necessary to explore if having more than 20 epochs gives a better result.
⏩ Main observations and risk
- The image dataset needs to be reviewed and ensure acceptable quality.
- The company needs also to explore the possibility to get more images, paying special attention to Neutral and Sad classes.
He, H., & Ma, Y. (Eds.). (2013). Imbalanced learning: foundations, algorithms, and applications.
Save, serialize, and export models | TensorFlow Core. (n.d.). TensorFlow. https://www.tensorflow.org/guide/keras/serialization_and_saving
Team, K. (n.d.). Keras documentation: Keras API reference. Keras API Reference. https://keras.io/api/
Module: tf | TensorFlow v2.13.0. (n.d.). TensorFlow. https://www.tensorflow.org/api_docs/python/tf
Colab Subscription Pricing. (n.d.). Colab Subscription Pricing. https://colab.research.google.com/signup